Is there a command I can invoke which will count the lines changed by a specific author in a Git repository? I know that there must be ways to count the number of commits as Github does this for their Impact graph.
This question is related to
git
command-line
statistics
repository
The answer is
I provided a modification of a short answer above, but it wasnt sufficient for my needs. I needed to be able to categorize both committed lines and lines in the final code. I also wanted a break down by file. This code does not recurse, it will only return the results for a single directory, but it is a good start if someone wanted to go further. Copy and paste into a file and make executable or run it with Perl.
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
my $dir = shift;
die "Please provide a directory name to check\n"
unless $dir;
chdir $dir
or die "Failed to enter the specified directory '$dir': $!\n";
if ( ! open(GIT_LS,'-|','git ls-files') ) {
die "Failed to process 'git ls-files': $!\n";
}
my %stats;
while (my $file = <GIT_LS>) {
chomp $file;
if ( ! open(GIT_LOG,'-|',"git log --numstat $file") ) {
die "Failed to process 'git log --numstat $file': $!\n";
}
my $author;
while (my $log_line = <GIT_LOG>) {
if ( $log_line =~ m{^Author:\s*([^<]*?)\s*<([^>]*)>} ) {
$author = lc($1);
}
elsif ( $log_line =~ m{^(\d+)\s+(\d+)\s+(.*)} ) {
my $added = $1;
my $removed = $2;
my $file = $3;
$stats{total}{by_author}{$author}{added} += $added;
$stats{total}{by_author}{$author}{removed} += $removed;
$stats{total}{by_author}{total}{added} += $added;
$stats{total}{by_author}{total}{removed} += $removed;
$stats{total}{by_file}{$file}{$author}{added} += $added;
$stats{total}{by_file}{$file}{$author}{removed} += $removed;
$stats{total}{by_file}{$file}{total}{added} += $added;
$stats{total}{by_file}{$file}{total}{removed} += $removed;
}
}
close GIT_LOG;
if ( ! open(GIT_BLAME,'-|',"git blame -w $file") ) {
die "Failed to process 'git blame -w $file': $!\n";
}
while (my $log_line = <GIT_BLAME>) {
if ( $log_line =~ m{\((.*?)\s+\d{4}} ) {
my $author = $1;
$stats{final}{by_author}{$author} ++;
$stats{final}{by_file}{$file}{$author}++;
$stats{final}{by_author}{total} ++;
$stats{final}{by_file}{$file}{total} ++;
$stats{final}{by_file}{$file}{total} ++;
}
}
close GIT_BLAME;
}
close GIT_LS;
print "Total lines committed by author by file\n";
printf "%25s %25s %8s %8s %9s\n",'file','author','added','removed','pct add';
foreach my $file (sort keys %{$stats{total}{by_file}}) {
printf "%25s %4.0f%%\n",$file
,100*$stats{total}{by_file}{$file}{total}{added}/$stats{total}{by_author}{total}{added};
foreach my $author (sort keys %{$stats{total}{by_file}{$file}}) {
next if $author eq 'total';
if ( $stats{total}{by_file}{$file}{total}{added} ) {
printf "%25s %25s %8d %8d %8.0f%%\n",'', $author,@{$stats{total}{by_file}{$file}{$author}}{qw{added removed}}
,100*$stats{total}{by_file}{$file}{$author}{added}/$stats{total}{by_file}{$file}{total}{added};
} else {
printf "%25s %25s %8d %8d\n",'', $author,@{$stats{total}{by_file}{$file}{$author}}{qw{added removed}} ;
}
}
}
print "\n";
print "Total lines in the final project by author by file\n";
printf "%25s %25s %8s %9s %9s\n",'file','author','final','percent', '% of all';
foreach my $file (sort keys %{$stats{final}{by_file}}) {
printf "%25s %4.0f%%\n",$file
,100*$stats{final}{by_file}{$file}{total}/$stats{final}{by_author}{total};
foreach my $author (sort keys %{$stats{final}{by_file}{$file}}) {
next if $author eq 'total';
printf "%25s %25s %8d %8.0f%% %8.0f%%\n",'', $author,$stats{final}{by_file}{$file}{$author}
,100*$stats{final}{by_file}{$file}{$author}/$stats{final}{by_file}{$file}{total}
,100*$stats{final}{by_file}{$file}{$author}/$stats{final}{by_author}{total}
;
}
}
print "\n";
print "Total lines committed by author\n";
printf "%25s %8s %8s %9s\n",'author','added','removed','pct add';
foreach my $author (sort keys %{$stats{total}{by_author}}) {
next if $author eq 'total';
printf "%25s %8d %8d %8.0f%%\n",$author,@{$stats{total}{by_author}{$author}}{qw{added removed}}
,100*$stats{total}{by_author}{$author}{added}/$stats{total}{by_author}{total}{added};
};
print "\n";
print "Total lines in the final project by author\n";
printf "%25s %8s %9s\n",'author','final','percent';
foreach my $author (sort keys %{$stats{final}{by_author}}) {
printf "%25s %8d %8.0f%%\n",$author,$stats{final}{by_author}{$author}
,100*$stats{final}{by_author}{$author}/$stats{final}{by_author}{total};
}
This script here will do it. Put it into authorship.sh, chmod +x it, and you're all set.
#!/bin/sh
declare -A map
while read line; do
if grep "^[a-zA-Z]" <<< "$line" > /dev/null; then
current="$line"
if [ -z "${map[$current]}" ]; then
map[$current]=0
fi
elif grep "^[0-9]" <<<"$line" >/dev/null; then
for i in $(cut -f 1,2 <<< "$line"); do
map[$current]=$((map[$current] + $i))
done
fi
done <<< "$(git log --numstat --pretty="%aN")"
for i in "${!map[@]}"; do
echo -e "$i:${map[$i]}"
done | sort -nr -t ":" -k 2 | column -t -s ":"
After looking at Alex's and Gerty3000's answer, I have tried to shorten the one-liner:
Basically, using git log numstat and not keeping track of the number of files changed.
Git version 2.1.0 on Mac OSX:
git log --format='%aN' | sort -u | while read name; do echo -en "$name\t"; git log --author="$name" --pretty=tformat: --numstat | awk '{ add += $1; subs += $2; loc += $1 - $2 } END { printf "added lines: %s, removed lines: %s, total lines: %s\n", add, subs, loc }' -; done
Example:
Jared Burrows added lines: 6826, removed lines: 2825, total lines: 4001
@mmrobins @AaronM @ErikZ @JamesMishra provided variants that all have an problem in common: they ask git to produce a mixture of info not intended for script consumption, including line contents from repository on the same line, then match the mess with a regexp.
This is a problem when some lines aren't valid UTF-8 text, and also when some lines happen to match the regexp (this happened here).
Here's a modified line that doesn't have these problems. It requests git to output data cleanly on separate lines, which makes it easy to filter what we want robustly:
git ls-files -z | xargs -0n1 git blame -w --line-porcelain | grep -a "^author " | sort -f | uniq -c | sort -n
You can grep for other strings, like author-mail, committer, etc.
Perhaps first do export LC_ALL=C (assuming bash) to force byte-level processing (this also happens to speed up grep tremendously from the UTF-8-based locales).
I wrote this Perl script to accomplish that task.
#!/usr/bin/env perl
use strict;
use warnings;
# save the args to pass to the git log command
my $ARGS = join(' ', @ARGV);
#get the repo slug
my $NAME = _get_repo_slug();
#get list of authors
my @authors = _get_authors();
my ($projectFiles, $projectInsertions, $projectDeletions) = (0,0,0);
#for each author
foreach my $author (@authors) {
my $command = qq{git log $ARGS --author="$author" --oneline --shortstat --no-merges};
my ($files, $insertions, $deletions) = (0,0,0);
my @lines = `$command`;
foreach my $line (@lines) {
if ($line =~ m/^\s(\d+)\s\w+\s\w+,\s(\d+)\s\w+\([\+|\-]\),\s(\d+)\s\w+\([\+|\-]\)$|^\s(\d+)\s\w+\s\w+,\s(\d+)\s\w+\(([\+|\-])\)$/) {
my $lineFiles = $1 ? $1 : $4;
my $lineInsertions = (defined $6 && $6 eq '+') ? $5 : (defined $2) ? $2 : 0;
my $lineDeletions = (defined $6 && $6 eq '-') ? $5 : (defined $3) ? $3 : 0;
$files += $lineFiles;
$insertions += $lineInsertions;
$deletions += $lineDeletions;
$projectFiles += $lineFiles;
$projectInsertions += $lineInsertions;
$projectDeletions += $lineDeletions;
}
}
if ($files || $insertions || $deletions) {
printf(
"%s,%s,%s,+%s,-%s,%s\n",
$NAME,
$author,
$files,
$insertions,
$deletions,
$insertions - $deletions
);
}
}
printf(
"%s,%s,%s,+%s,-%s,%s\n",
$NAME,
'PROJECT_TOTAL',
$projectFiles,
$projectInsertions,
$projectDeletions,
$projectInsertions - $projectDeletions
);
exit 0;
#get the remote.origin.url joins that last two pieces (project and repo folder)
#and removes any .git from the results.
sub _get_repo_slug {
my $get_remote_url = "git config --get remote.origin.url";
my $remote_url = `$get_remote_url`;
chomp $remote_url;
my @parts = split('/', $remote_url);
my $slug = join('-', @parts[-2..-1]);
$slug =~ s/\.git//;
return $slug;
}
sub _get_authors {
my $git_authors = 'git shortlog -s | cut -c8-';
my @authors = `$git_authors`;
chomp @authors;
return @authors;
}
I named it git-line-changes-by-author and put into /usr/local/bin. Because it is saved in my path, I can issue the command git line-changes-by-author --before 2018-12-31 --after 2020-01-01 to get the report for the 2019 year. As an example. And if I were to misspell the name git will suggest the proper spelling.
You may want to adjust the _get_repo_slug sub to only include the last portion of the remote.origin.url as my repos are saved as project/repo and your might not be.
The Answer from AaronM using the shell one-liner is good, but actually, there is yet another bug, where spaces will corrupt the user names if there are different amounts of white spaces between the user name and the date. The corrupted user names will give multiple rows for user counts and you have to sum them up yourself.
This small change fixed the issue for me:
git ls-files -z | xargs -0n1 git blame -w --show-email | perl -n -e '/^.*?\((.*?)\s+[\d]{4}/; print $1,"\n"' | sort -f | uniq -c | sort -n
Notice the + after \s which will consume all whitespaces from the name to the date.
Actually adding this answer as much for my own rememberance as for helping anyone else, since this is at least the second time I google the subject :)
- Edit 2019-01-23 Added
--show-emailtogit blame -wto aggregate on email instead, since some people use differentNameformats on different computers, and sometimes two people with the same name are working in the same git.
In addition to Charles Bailey's answer, you might want to add the -C parameter to the commands. Otherwise file renames count as lots of additions and removals (as many as the file has lines), even if the file content was not modified.
To illustrate, here is a commit with lots of files being moved around from one of my projects, when using the git log --oneline --shortstat command:
9052459 Reorganized project structure
43 files changed, 1049 insertions(+), 1000 deletions(-)
And here the same commit using the git log --oneline --shortstat -C command which detects file copies and renames:
9052459 Reorganized project structure
27 files changed, 134 insertions(+), 85 deletions(-)
In my opinion the latter gives a more realistic view of how much impact a person has had on the project, because renaming a file is a much smaller operation than writing the file from scratch.
I found the following to be useful to see who had the most lines that were currently in the code base:
git ls-files -z | xargs -0n1 git blame -w | ruby -n -e '$_ =~ /^.*\((.*?)\s[\d]{4}/; puts $1.strip' | sort -f | uniq -c | sort -n
The other answers have mostly focused on lines changed in commits, but if commits don't survive and are overwritten, they may just have been churn. The above incantation also gets you all committers sorted by lines instead of just one at a time. You can add some options to git blame (-C -M) to get some better numbers that take file movement and line movement between files into account, but the command might run a lot longer if you do.
Also, if you're looking for lines changed in all commits for all committers, the follow little script is helpful:
Save your logs into file using:
git log --author="<authorname>" --oneline --shortstat > logs.txt
For Python lovers:
with open(r".\logs.txt", "r", encoding="utf8") as f:
files = insertions = deletions = 0
for line in f:
if ' changed' in line:
line = line.strip()
spl = line.split(', ')
if len(spl) > 0:
files += int(spl[0].split(' ')[0])
if len(spl) > 1:
insertions += int(spl[1].split(' ')[0])
if len(spl) > 2:
deletions += int(spl[2].split(' ')[0])
print(str(files).ljust(10) + ' files changed')
print(str(insertions).ljust(10) + ' insertions')
print(str(deletions).ljust(10) + ' deletions')
Your outputs would be like:
225 files changed
6751 insertions
1379 deletions
In case anyone wants to see the stats for every user in their codebase, a couple of my coworkers recently came up with this horrific one-liner:
git log --shortstat --pretty="%cE" | sed 's/\(.*\)@.*/\1/' | grep -v "^$" | awk 'BEGIN { line=""; } !/^ / { if (line=="" || !match(line, $0)) {line = $0 "," line }} /^ / { print line " # " $0; line=""}' | sort | sed -E 's/# //;s/ files? changed,//;s/([0-9]+) ([0-9]+ deletion)/\1 0 insertions\(+\), \2/;s/\(\+\)$/\(\+\), 0 deletions\(-\)/;s/insertions?\(\+\), //;s/ deletions?\(-\)//' | awk 'BEGIN {name=""; files=0; insertions=0; deletions=0;} {if ($1 != name && name != "") { print name ": " files " files changed, " insertions " insertions(+), " deletions " deletions(-), " insertions-deletions " net"; files=0; insertions=0; deletions=0; name=$1; } name=$1; files+=$2; insertions+=$3; deletions+=$4} END {print name ": " files " files changed, " insertions " insertions(+), " deletions " deletions(-), " insertions-deletions " net";}'
(Takes a few minutes to crunch through our repo, which has around 10-15k commits.)
The best tool so far I identfied is gitinspector. It give the set report per user, per week etc You can install like below with npm
npm install -g gitinspector
The links to get the more details
https://www.npmjs.com/package/gitinspector
https://github.com/ejwa/gitinspector/wiki/Documentation
https://github.com/ejwa/gitinspector
example commands are
gitinspector -lmrTw
gitinspector --since=1-1-2017 etc
A solution was given with ruby in the middle, perl being a little more available by default here is an alternative using perl for current lines by author.
git ls-files -z | xargs -0n1 git blame -w | perl -n -e '/^.*\((.*?)\s*[\d]{4}/; print $1,"\n"' | sort -f | uniq -c | sort -n
You want Git blame.
There's a --show-stats option to print some, well, stats.
git-fame
https://github.com/oleander/git-fame-rb
This is a nice tool to get the count for all authors at once, including commit and modified files count:
sudo apt-get install ruby-dev
sudo gem install git_fame
cd /path/to/gitdir && git fame
There is also Python version at https://github.com/casperdcl/git-fame (mentioned by @fracz):
sudo apt-get install python-pip python-dev build-essential
pip install --user git-fame
cd /path/to/gitdir && git fame
Sample output:
Total number of files: 2,053
Total number of lines: 63,132
Total number of commits: 4,330
+------------------------+--------+---------+-------+--------------------+
| name | loc | commits | files | percent |
+------------------------+--------+---------+-------+--------------------+
| Johan Sørensen | 22,272 | 1,814 | 414 | 35.3 / 41.9 / 20.2 |
| Marius Mathiesen | 10,387 | 502 | 229 | 16.5 / 11.6 / 11.2 |
| Jesper Josefsson | 9,689 | 519 | 191 | 15.3 / 12.0 / 9.3 |
| Ole Martin Kristiansen | 6,632 | 24 | 60 | 10.5 / 0.6 / 2.9 |
| Linus Oleander | 5,769 | 705 | 277 | 9.1 / 16.3 / 13.5 |
| Fabio Akita | 2,122 | 24 | 60 | 3.4 / 0.6 / 2.9 |
| August Lilleaas | 1,572 | 123 | 63 | 2.5 / 2.8 / 3.1 |
| David A. Cuadrado | 731 | 111 | 35 | 1.2 / 2.6 / 1.7 |
| Jonas Ängeslevä | 705 | 148 | 51 | 1.1 / 3.4 / 2.5 |
| Diego Algorta | 650 | 6 | 5 | 1.0 / 0.1 / 0.2 |
| Arash Rouhani | 629 | 95 | 31 | 1.0 / 2.2 / 1.5 |
| Sofia Larsson | 595 | 70 | 77 | 0.9 / 1.6 / 3.8 |
| Tor Arne Vestbø | 527 | 51 | 97 | 0.8 / 1.2 / 4.7 |
| spontus | 339 | 18 | 42 | 0.5 / 0.4 / 2.0 |
| Pontus | 225 | 49 | 34 | 0.4 / 1.1 / 1.7 |
+------------------------+--------+---------+-------+--------------------+
But be warned: as mentioned by Jared in the comment, doing it on a very large repository will take hours. Not sure if that could be improved though, considering that it must process so much Git data.
this is the best way and it also gives you a clear picture of total number of commits by all the user
git shortlog -s -n
This gives some statistics about the author, modify as required.
Using Gawk:
git log --author="_Your_Name_Here_" --pretty=tformat: --numstat \
| gawk '{ add += $1; subs += $2; loc += $1 - $2 } END { printf "added lines: %s removed lines: %s total lines: %s\n", add, subs, loc }' -
Using Awk on Mac OSX:
git log --author="_Your_Name_Here_" --pretty=tformat: --numstat | awk '{ add += $1; subs += $2; loc += $1 - $2 } END { printf "added lines: %s, removed lines: %s, total lines: %s\n", add, subs, loc }' -
EDIT (2017)
There is a new package on github that looks slick and uses bash as dependencies (tested on linux). It's more suitable for direct usage rather than scripts.
It's git-quick-stats (github link).
Copy git-quick-stats to a folder and add the folder to path.
mkdir ~/source
cd ~/source
git clone [email protected]:arzzen/git-quick-stats.git
mkdir ~/bin
ln -s ~/source/git-quick-stats/git-quick-stats ~/bin/git-quick-stats
chmod +x ~/bin/git-quick-stats
export PATH=${PATH}:~/bin
Usage:
git-quick-stats
To count number of commits by a given author (or all authors) on a given branch you can use git-shortlog; see especially its --numbered and --summary options, e.g. when run on git repository:
$ git shortlog v1.6.4 --numbered --summary
6904 Junio C Hamano
1320 Shawn O. Pearce
1065 Linus Torvalds
692 Johannes Schindelin
443 Eric Wong
For windows users you can use following batch script that counts added/removed lines for specified author
@echo off
set added=0
set removed=0
for /f "tokens=1-3 delims= " %%A in ('git log --pretty^=tformat: --numstat --author^=%1') do call :Count %%A %%B %%C
@echo added=%added%
@echo removed=%removed%
goto :eof
:Count
if NOT "%1" == "-" set /a added=%added% + %1
if NOT "%2" == "-" set /a removed=%removed% + %2
goto :eof
https://gist.github.com/zVolodymyr/62e78a744d99d414d56646a5e8a1ff4f
The question asked for information on a specific author, but many of the answers were solutions that returned ranked lists of authors based on their lines of code changed.
This was what I was looking for, but the existing solutions were not quite perfect. In the interest of people that may find this question via Google, I've made some improvements on them and made them into a shell script, which I display below.
There are no dependencies on either Perl or Ruby. Furthermore, whitespace, renames, and line movements are taken into account in the line change count. Just put this into a file and pass your Git repository as the first parameter.
#!/bin/bash
git --git-dir="$1/.git" log > /dev/null 2> /dev/null
if [ $? -eq 128 ]
then
echo "Not a git repository!"
exit 128
else
echo -e "Lines | Name\nChanged|"
git --work-tree="$1" --git-dir="$1/.git" ls-files -z |\
xargs -0n1 git --work-tree="$1" --git-dir="$1/.git" blame -C -M -w |\
cut -d'(' -f2 |\
cut -d2 -f1 |\
sed -e "s/ \{1,\}$//" |\
sort |\
uniq -c |\
sort -nr
fi
Here is a great repo that makes your life easier
git-quick-stats
On a mac with brew installed
brew install git-quick-stats
Run
git-quick-stats
Just choose which option you want from this list by typing in the number listed and hitting enter.
Generate:
1) Contribution stats (by author)
2) Contribution stats (by author) on a specific branch
3) Git changelogs (last 10 days)
4) Git changelogs by author
5) My daily status
6) Save git log output in JSON format
List:
7) Branch tree view (last 10)
8) All branches (sorted by most recent commit)
9) All contributors (sorted by name)
10) Git commits per author
11) Git commits per date
12) Git commits per month
13) Git commits per weekday
14) Git commits per hour
15) Git commits by author per hour
Suggest:
16) Code reviewers (based on git history)
Here's a quick ruby script that corrals up the impact per user against a given log query.
For example, for rubinius:
Brian Ford: 4410668
Evan Phoenix: 1906343
Ryan Davis: 855674
Shane Becker: 242904
Alexander Kellett: 167600
Eric Hodel: 132986
Dirkjan Bussink: 113756
...
the script:
#!/usr/bin/env ruby
impact = Hash.new(0)
IO.popen("git log --pretty=format:\"%an\" --shortstat #{ARGV.join(' ')}") do |f|
prev_line = ''
while line = f.gets
changes = /(\d+) insertions.*(\d+) deletions/.match(line)
if changes
impact[prev_line] += changes[1].to_i + changes[2].to_i
end
prev_line = line # Names are on a line of their own, just before the stats
end
end
impact.sort_by { |a,i| -i }.each do |author, impact|
puts "#{author.strip}: #{impact}"
end
you can use whodid (https://www.npmjs.com/package/whodid)
$ npm install whodid -g
$ cd your-project-dir
and
$ whodid author --include-merge=false --path=./ --valid-threshold=1000 --since=1.week
or just type
$ whodid
then you can see result like this
Contribution state
=====================================================
score | author
-----------------------------------------------------
3059 | someguy <[email protected]>
585 | somelady <[email protected]>
212 | niceguy <[email protected]>
173 | coolguy <[email protected]>
=====================================================
Here's a short one-liner that produces stats for all authors. It's much faster than Dan's solution above at https://stackoverflow.com/a/20414465/1102119 (mine has time complexity O(N) instead of O(NM) where N is the number of commits, and M the number of authors).
git log --no-merges --pretty=format:%an --numstat | awk '/./ && !author { author = $0; next } author { ins[author] += $1; del[author] += $2 } /^$/ { author = ""; next } END { for (a in ins) { printf "%10d %10d %10d %s\n", ins[a] - del[a], ins[a], del[a], a } }' | sort -rn
Source: Stackoverflow.com