2015-11-26

Perl script to backup GitLab.com project issues.

In using the GitLab.com hosting service, I could not find an archive button to download the issues for a project. If such functionality exists, it is well hidden from me.

It does have a button to download a zip of the git repos itself, but I find this incredibly bizarre, since if I'm using GitLab as a git remote, that means I already have the repos.

What I wrote is only meant to be a primitive last resort backup. To manually reenter all my issues from the saved output of this script would be a giant pain in the ass, but at least recovery would be possible. Before, if all my issues magically disappeared from GitLab.com, accurate restoration would be impossible.

You'll have to install GitLab::API::v3 module. Right now, it doesn't handle issue comments, I had to add the issue_comments method myself. See bottom of this post that.

In any case, here is:

use v5.18;
use warnings;
use strict;

use Data::Dumper;
use GitLab::API::v3;
use POSIX qw(strftime);

my $v3_api_url = 'http://gitlab.com/api/v3';
my $token = 'YeAhR1GhTkEePgUeSs1nG';

my $api = GitLab::API::v3->new(
url   => $v3_api_url,
token => $token,
);

my $project_id_num = 123456;

say "Getting issues for project $project_id_num.";
my $all_issues = $api->paginator(
    'issues',
    $project_id_num,
)->all();

my $backup_root = '/Users/jeff/VGT/GitLab backup/';
my $now_string = strftime("%Y-%m-%d_%H-%M-%S", localtime);
my $backup_dir = $backup_root . $now_string;
mkdir $backup_dir or die "Fail to mkdir $backup_dir";
my $issues_out = $backup_dir . '/issues.dump';

say "Logging issues.";
open (my $issues_fh, '>', $issues_out) or die "Cannot open $issues_out";
print $issues_fh Dumper($all_issues);
close $issues_fh;

say "Getting comments.";
for my $issue_ref (@$all_issues) {
    my $issue_id = $issue_ref->{'id'};
    my $iid = $issue_ref->{'iid'};
    my $comments = $api->issue_comments($project_id_num, $issue_id);
    
    next if scalar(@$comments) == 0;
    say "Logging comments for $iid.";
    
    my $comment_dir = $backup_dir . '/' . $iid;
    mkdir $comment_dir or die "Fail to mkdir $comment_dir";
    
    my $comment_out = $comment_dir . '/comments.dump';
    open my $comment_fh, '>', $comment_out or die "Cannot open $comment_out";
    print $comment_fh Dumper($comments);
    close $comment_fh;
}

say "Logging labels.";
my $labels = $api->labels($project_id_num);
my $labels_out = $backup_dir . '/labels.dump';
open my $labels_fh, '>', $labels_out or die "Cannot open $labels_out";
print $labels_fh Dumper($labels);
close $labels_fh;

exit 0;


This is what I added to wherever-your-library-is/GitLab/API/v3.pm. Here is git diff output:

$ git diff 1c3c02b64f3a3c608891bc3021a002892b96958e v3.pm 
diff --git a/v3.pm b/v3.pm
index bd745c0..b8effea 100644
--- a/v3.pm
+++ b/v3.pm
@@ -1866,6 +1866,27 @@ sub edit_issue {
     return $self->put( $path, ( defined($params) ? $params : () ) );
 }
 
+=head2 issue_comments
+ 
+    my $comments = $api->issue_comments(
+        $project_id,
+        $issue_id,
+     );
+ 
+ Sends a Clt;GET> request to Clt;/projects/:project_id/issues/:issue_id/notes> and returns the decoded/deserialized response body.
+ 
+=cut
+
+sub issue_comments {
+    my $self = shift;
+    croak 'issue_comments must be called with 2 arguments' if @_ != 2;
+    croak 'The #1 argument ($project_id) to issue_comments must be a scalar' if ref($_[0]) or (!defined $_[0]);
+    croak 'The #2 argument ($$issue_id) to issue_comments must be a scalar' if ref($_[1]) or (!defined $_[1]);
+    my $path = sprintf('/projects/%s/issues/%s/notes', (map { uri_escape($_) } @_));
+    $log->infof( 'Making %s request against %s with params %s.', 'GET', $path, undef );
+    return $self->get( $path );
+}
+
 =head1 LABEL METHODS
 
 See L<http://doc.gitlab.com/ce/api/labels.html>.

And that's it!

1 comment:

  1. Thanks for sharing! We plan to add export and import of projects to GitLab https://gitlab.com/gitlab-org/gitlab-ce/issues/3050 ^Sid

    ReplyDelete