git utility to move/rename file or folder and retain history with it.

git-mv-with-history

#!/bin/bash
#
# git-mv-with-history -- move/rename file or folder, with history.
#
# Moving a file in git doesn't track history, so the purpose of this
# utility is best explained from the kernel wiki:
#
#   Git has a rename command git mv, but that is just for convenience.
#   The effect is indistinguishable from removing the file and adding another
#   with different name and the same content.
#
# https://git.wiki.kernel.org/index.php/GitFaq#Why_does_Git_not_.22track.22_renames.3F
#
# While the above sucks, git has the ability to let you rewrite history
# of anything via `filter-branch`. This utility just wraps that functionality,
# but also allows you to easily specify more than one rename/move at a
# time (since the `filter-branch` can be slow on big repos).
#
# Usage:
#
#   git-rewrite-history [-d/--dry-run] [-v/--verbose] <srcname>=<destname> <...> <...>
#
# After the repsitory is re-written, eyeball it, commit and push up.
#
# Given this example repository structure:
#
#   src/makefile
#   src/test.cpp
#   src/test.h
#   src/help.txt
#   README.txt
#
# The command:
#
#   git-rewrite-history README.txt=README.md  \     <-- rename to markdpown
#                       src/help.txt=docs/    \     <-- move help.txt into docs
#                       src/makefile=src/Makefile   <-- capitalize makefile
#
#  Would restructure and retain history, resulting in the new structure:
#
#    docs/help.txt
#    src/Makefile
#    src/test.cpp
#    src/test.h
#    README.md
#
# @author emiller
# @date   2013-09-29
#

function usage() {
  echo "usage: `basename $0` [-d/--dry-run] [-v/--verbose] <srcname>=<destname> <...> <...>"
  [ -z "$1" ] || echo $1

  exit 1
}

[ ! -d .git ] && usage "error: must be ran from within the root of the repository"

dryrun=0
filter=""
verbose=""
repo=$(basename `git rev-parse --show-toplevel`)

while [[ $1 =~ ^\- ]]; do
  case $1 in
    -d|--dry-run)
      dryrun=1
      ;;

    -v|--verbose)
      verbose="-v"
      ;;

    *)
      usage "invalid argument: $1"
  esac

  shift
done

for arg in $@; do
  val=`echo $arg | grep -q '=' && echo 1 || echo 0`
  src=`echo $arg | sed 's/\(.*\)=\(.*\)/\1/'`
  dst=`echo $arg | sed 's/\(.*\)=\(.*\)/\2/'`
  dir=`echo $dst | grep -q '/$' && echo $dst || dirname $dst`

  [ "$val" -ne 1  ] && usage
  [ ! -e "$src"   ] && usage "error: $src does not exist"

  filter="$filter                            \n\
    if [ -e \"$src\" ]; then                 \n\
      echo                                   \n\
      if [ ! -e \"$dir\" ]; then             \n\
        mkdir -p ${verbose} \"$dir\" && echo \n\
      fi                                     \n\
      mv $verbose \"$src\" \"$dst\"          \n\
    fi                                       \n\
  "
done

[ -z "$filter" ] && usage

if [[ $dryrun -eq 1 || ! -z $verbose ]]; then
  echo
  echo "tree-filter to execute against $repo:"
  echo -e "$filter"
fi

[ $dryrun -eq 0 ] && git filter-branch -f --tree-filter "`echo -e $filter`"

Thanks, works nicely (though I too had to remove the "-e").

@docbill hi, have you some details on the method you used on big repo ?
@emiller is there some "range" filter possible ? Or limit to a branch only ?

Cannot merge into a branch after running the script:
xxx and yyy are entirely different commit histories.

git log --follow is easier solution than this script, in my opinion.

Unfortunately git log --follow is not a default position. So, if you are merging multiple repos into 1, then EVERY file is renamed in that process. This mechanism rewrites the history so that it was ALWAYS in the new location.

Thanks. it helps me a lot.

Is there a way to clean up the renames, essentially it's making my repository much bigger than it should be, I posted something on stackoverflow trying to get some help with the same question.

https://stackoverflow.com/questions/54318225/remove-history-for-files-not-in-masterhead

After doing a rename, I think that I should mention you should merge current state and then follow something like the following to ensure that the directory get's delete throughout history after the rename.

https://stackoverflow.com/questions/10067848/remove-folder-and-its-contents-from-git-githubs-history

Thank you! The code worked.

I downloaded the zip file, extracted the script file, renamed it to git-mv-with-history and pasted it in the root folder of the repository. The I ran it with the following command:

bash git-mv-with-history old_file_name=new_file_name

Then, I had to forcefully push changes to my Github repository with:

git push -f

and hurry, it had worked perfectly.

I have 70600+ commit logs, when i run 'bash git-mv-with-history old_file_name=new_file_name', it tasks too much hours to overwrite all the logs.Do you have some ideas to descrease time？Thank you

@xiaoliuTX maybe look at https://gist.github.com/ilanKeshet/bf4251b21919d341cf4431f89e77a8a5#file-git-mv-with-history

Works like magic. I used this script to rename my entire src directory under my project tree. Thanks!

Great script!! I have used it to move folders and files both.
I was getting the same error as mentioned by @Yashrajsingh but it was fixed by the solution provided by him - https://gist.github.com/emiller/6769886#gistcomment-2296722

I find this script very useful. Moved some tracked files to different directories w/o losing the files' Git logs.

However, when I tried to move a file from one directory to another, it didn't quite complete the move.

I get this message: "WARNING: Ref 'refs/heads/master' is unchanged"

Then Git shows the file has been moved to the new directory but Git status shows a pending "Deleted" status of the file in the old directory.

BTW, the Git repo had no pending commits for any of the tracked files when I invoked this script.

If I commit the changes that this script made to Git, the file in the new directory has no logs anymore for some reason. So I backed out the change by doing a git reset --hard <prev. commit hash>

Has anyone else encountered this problem?

From what I understand this script should also move folders with files, when I run it bash git-mv-with-history src=apps\product-catalog\ this error is displayed:

git-mv-with-history: line 63: git: command not found
BusyBox v1.29.3 (2019-01-24 07:45:07 UTC) multi-call binary.

Usage: basename FILE [SUFFIX]

Strip directory path and .SUFFIX from FILE
git-mv-with-history: line 110: git: command not found

git: 'rewrite-history' is not a git command. See 'git --help'.

This script knows its job. Thank you!

I have been looking for this kind of script. We have some SQL scripts stored as ".txt" files in many directories. I am new to bash script. Is there any way I can rename all files with '.txt' extension to '.sql' in batch?

Thanks for this script! However, it's worth noting that Git now officially recommends against using git filter-branch due to performance and safety concerns. From the official Git documentation:

WARNING: git filter-branch has a plethora of pitfalls that can produce non-obvious manglings of the intended history rewrite [...] Please use an alternative history filtering tool such as git filter-repo.

I've created a PowerShell prototype that uses git-filter-repo instead. It's still experimental but aims to:

• Work cross-platform (Windows/Linux/macOS)
• Use Git's built-in rename detection
• Preserve file history properly using git-filter-repo

Would love to get feedback and suggestions for improvement if anyone wants to test it out!

Anything like this with a date of 2013-09-29, I could consider more inspiration, not something one should consider using. Although, you can probably just drop it in chatgpt with a copy of this discussion, and chatgpt will output something ready to review and then debug.