Skip to content

Instantly share code, notes, and snippets.

@jonmagic
Last active December 16, 2024 12:17
Show Gist options
  • Save jonmagic/5282384165e0f86ef105 to your computer and use it in GitHub Desktop.
Save jonmagic/5282384165e0f86ef105 to your computer and use it in GitHub Desktop.
Complete issue import API walkthrough with curl

This gist describes a new GitHub API for importing issues. This API hasn't been finalized yet, but when it is -- it will be announced on the official API blog.

Overview:

General notes

To access the Issue Import API, you'll need to include application/vnd.github.golden-comet-preview in your request's Accept header. You'll also need to have admin permissions on the repository.

The maximum request size is 1MB.

Issues and comments created with this API do not trigger any notifications to mentioned users or users watching the repository into which issues are being imported. This is unlike the Issues API which does trigger notifications).

Like all other API requests, requests made to the Issue Import API are affected by regular API rate limits and abuse rate limits.

Issue imports started with this API are processed in the background asynchronously. When you start an import, you get back an import URL which you can use to check if the import completed, and whether it completed successfully or not. Because of this background processing, issue numbers are handed out to successfully imported issues once the processing is completed, and not when you start an import. As a result, if you use this API to submit two imports for issues A and B, and B is submitted after A, but before A's import completes, then it is possible that B will have a lower issue number than A. If you need to make sure that issues have specific numbers, then the recommended approach is that you start an import, wait for that import to complete and make sure it completed successfully, and only after that start a new import.

If you have any feedback or questions about the Issue Import API, please get in touch.

Create a repository

Create a repository using the web interface or the API.

Here is an example API request to create a repository:

curl -X POST -H "Authorization: token ${GITHUB_TOKEN}" \
  -d '{"name":"foo","private":true}' \
  https://api.github.com/user/repos

Try to start an issue import with incomplete data

The Issue Import API attempts to provide actionable feedback when a request does not meet validation criteria.

Request

curl -X POST -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  -d '{"issue":{"title":"My money, mo issues"}}' \
  https://api.github.com/repos/${GITHUB_USERNAME}/foo/import/issues

Response

{
  "message": "Validation Failed",
  "errors": [
    {
      "resource": "Issue",
      "code": "missing_field",
      "field": "body"
    }
  ],
  "documentation_url": "https://developer.github.com/v3"
}

Start an issue import

Just make a post request to the import issues API endpoint for the repository and include the issue and comments json as data. An issue only requires the following fields:

{
  "issue": {
    "title": "Imported from some other system",
    "body": "..."
  }
}

And an issue with a comment only needs the comment body added:

{
  "issue": {
    "title": "Imported from some other system",
    "body": "..."
  },
  "comments": [
    {
      "body": "talk talk"
    }
  ]
}

Request

curl -X POST -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  -d '{"issue":{"title":"Fix broken widgets","body":"- [ ] widget 1\n- [ ] widget 2","created_at":"2014-03-16T18:21:16Z"},"comments":[{"body":"Can we wrap this up soon?","created_at":"2014-03-16T17:15:42Z"}]}' \
  https://api.github.com/repos/${GITHUB_USERNAME}/foo/import/issues

Response

{
  "id": 3,
  "status": "pending",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/3",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo"
}

Supported issue and comment fields

The complete json schema looks like this:

{
  "issue": {
    "title": "Imported from some other system",
    "body": "...",
    "created_at": "2014-01-01T12:34:58Z",
    "closed_at": "2014-01-02T12:24:56Z",
    "updated_at": "2014-01-03T11:34:53Z",
    "assignee": "jonmagic",
    "milestone": 1,
    "closed": true,
    "labels": [
      "bug",
      "low"
    ]
  },
  "comments": [
    {
      "created_at": "2014-01-02T12:34:56Z",
      "body": "talk talk"
    }
  ]
}

Note: The body of an issue or comment will be parsed and rendered using Markdown if the created_at value is set to a time after 2009-04-20T19:00:00Z, and using Textile if it's older. The body may be at most 65535 characters long. As of 2016-03-15 all newly created issues will be rendered with Markdown regardless of the created_at timestamp date.

Check status of issue import

After making the API request to import an issue you can check the status of the import by making a get request to the url value from the response in the previous post request.

Request

curl -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  https://api.github.com/repos/#{GITHUB_USERNAME}/foo/import/issues/3

Response

{
  "id": 3,
  "status": "imported",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/3",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo",
  "issue_url": "https://api.github.com/repos/jonmagic/foo/issues/1"
}

Check status of multiple issues

You can check the status of issues imported since a date or date and time.

Request

curl -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  https://api.github.com/repos/#{GITHUB_USERNAME}/foo/import/issues?since=2015-03-15

Response

[
  {
    "id": 3,
    "status": "imported",
    "url": "https://api.github.com/repos/jonmagic/foo/import/issues/3",
    "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
    "repository_url": "https://api.github.com/repos/jonmagic/foo",
    "issue_url": "https://api.github.com/repos/jonmagic/foo/issues/1"
  }
]

Import fails because issue has an invalid milestone, assignee, creator or label

What happens if you try to import an issue with an invalid attributes?

Import issue request

curl -X POST -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  -d '{"issue":{"title":"Fix broken widgets","body":"...","milestone":5,
      "assignee":"bobbyfoobar","labels":["bug","invalid,label"]}}' \
  https://api.github.com/repos/${GITHUB_USERNAME}/foo/import/issues

Import issue response

{
  "id": 7,
  "status": "pending",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/7",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo"
}

Import status request

curl -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  https://api.github.com/repos/#{GITHUB_USERNAME}/foo/import/issues/7

Import status response

{
  "id": 7,
  "status": "failed",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/7",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo",
  "created_at": "2015-03-18T19:40:53-07:00",
  "updated_at": "2015-03-18T19:40:58-07:00",
  "errors": [
    {
      "location": "/issue/milestone",
      "resource": "Issue",
      "field": "milestone",
      "value": "5",
      "code": "invalid"
    },
    {
      "location": "/issue/assignee",
      "resource": "Issue",
      "field": "assignee",
      "value": "bobbyfoobar",
      "code": "invalid"
    },
    {
      "location": "/issue/labels[1]",
      "resource": "Label",
      "field": "name",
      "value": "invalid,label",
      "code": "invalid"
    }
  ]
}

Import fails because of unexpected comment creation error

Import issue request

curl -X POST -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  -d '{"issue":{"title":"foo","body":"1"},"comments":[{"body":"fail this comment somehow"}]}' \
  https://api.github.com/repos/${GITHUB_USERNAME}/foo/import/issues

Import issue response

{
  "id": 12,
  "status": "pending",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/12",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo",
  "created_at": "2015-03-18T21:45:58-07:00",
}

Import status request

curl -H "Authorization: token ${GITHUB_TOKEN}" \
  -H "Accept: application/vnd.github.golden-comet-preview+json" \
  https://api.github.com/repos/#{GITHUB_USERNAME}/foo/import/issues/12

Import status response

{
  "id": 12,
  "status": "failed",
  "url": "https://api.github.com/repos/jonmagic/foo/import/issues/12",
  "import_issues_url": "https://api.github.com/repos/jonmagic/foo/import/issues",
  "repository_url": "https://api.github.com/repos/jonmagic/foo",
  "created_at": "2015-03-18T21:45:58-07:00",
  "updated_at": "2015-03-18T21:46:05-07:00",
  "issue_url": "https://api.github.com/repos/jonmagic/foo/issues/7",
  "errors": [
    {
      "location": "/comments[0]",
      "resource": "IssueComment",
      "field": null,
      "value": null,
      "code": "error"
    }
  ]
}
@Zimmi48
Copy link

Zimmi48 commented Oct 11, 2017

Testing this with:

$ curl -X POST -H "Authorization: token ${GITHUB_TOKEN}"   -H "Accept: application/vnd.github.golden-comet-preview+json"   -d '{"issue":{"title":"Fake issue","body":"Fake issue"}}'   https://api.github.com/repos/Zimmi48/test-issue-import-api/import/issues

gives me:

{
  "id": 1407016,
  "status": "pending",
  "url": "https://api.github.com/repos/Zimmi48/test-issue-import-api/import/issues/1407016",
  "import_issues_url": "https://api.github.com/repos/Zimmi48/test-issue-import-api/import/issues",
  "repository_url": "https://api.github.com/repos/Zimmi48/test-issue-import-api",
  "created_at": "2017-10-11T17:53:02+02:00",
  "updated_at": "2017-10-11T17:53:02+02:00"
}

But then

$ curl -H "Authorization: token ${GITHUB_TOKEN}"   -H "Accept: application/vnd.github.golden-comet-preview+json"  https://api.github.com/repos/Zimmi48/test-issue-import-api/import/issues/1407016

yields:

{
  "message": "Not Found",
  "documentation_url": "https://developer.github.com/v3"
}

and

$ curl -H "Authorization: token ${GITHUB_TOKEN}"   -H "Accept: application/vnd.github.golden-comet-preview+json"   https://api.github.com/repos/Zimmi48/test-issue-import-api/import/issues?since=2017-10-11

yields:

[

]

and sure enough: the issue is not created.

@Zimmi48
Copy link

Zimmi48 commented Oct 11, 2017

The issue was finally added but with a very important delay.

@Zimmi48
Copy link

Zimmi48 commented Oct 12, 2017

I don't know what was going on yesterday, but testing again now and this works perfectly and the API is very fast.

Copy link

ghost commented Oct 24, 2017

I experienced the same delay as well. Likely @github has some moderation.

@Zimmi48
Copy link

Zimmi48 commented Oct 26, 2017

@xoviat: No when I experienced it, it was due to server problems on their side (at least this is what their support told me). Later I was able to import lots of issues using this API without any problems.

@zzzeek
Copy link

zzzeek commented Feb 7, 2018

OK, this is bugging me. If i set an issue with closed-at and updated-at date, the labels still display as "zzzeek labeled the issue as foo just now". the issue was last updated in like 2006. Can this one little thing be fixed?

@DrewHood
Copy link

I noticed that the API will fail if I try to use a label on an issue that already exists in my project but isn’t an exact match for letter case. For example, Github’s default label “bug” conflicted with my pre-existing label “Bug” (note the capital B in mine). Deleting the lowercase default provided by Github allowed my issues with that label to be imported correctly. It’s a subtle thing but maybe the api should use a case-insensitive match?

@jeffwidman
Copy link

jeffwidman commented Jun 1, 2018

Please direct feedback to [email protected] as gist does not have comment notifications. Thanks!

Will do. But wanted to also post here in case anyone else knows offhand...

Is there a way to import attachments with this? They are supported in the GitHub web UI, just not seeing it in the API: jeffwidman/bitbucket-issue-migration#103

@AlexanderAmelkin
Copy link

@jonmagic, I've sent this to [email protected], but either I'm having some problems with my e-mail, or my message got lost because I haven't got any (even automatic) reply. Here is my report:

I'm trying to migrate from SourceForge tickets to GitHub issues.
I'm using the method described at https://gist.github.com/jonmagic/5282384165e0f86ef105
and I'm having a problem with it. Some tickets just won't import although from the server's
response it looks like they should have succeeded.

Specifically, this request looks like successful:

curl -X POST -H "Authorization: token `cat token`" \
     -H "Accept: application/vnd.github.golden-comet-preview+json" \
     -d @foo.json \
     https://api.github.com/repos/ipmitool/test/import/issues
{
  "id": 1745386,
  "status": "pending",
  "url": "https://api.github.com/repos/ipmitool/test/import/issues/1745386",
  "import_issues_url": "https://api.github.com/repos/ipmitool/test/import/issues",
  "repository_url": "https://api.github.com/repos/ipmitool/test",
  "created_at": "2018-06-06T17:06:35.660+03:00",
  "updated_at": "2018-06-06T17:06:35.660+03:00"
}

However if you check that "url" from the response, you will get this:

{
  "message": "Not Found",
  "documentation_url": "https://gist.github.com/jonmagic/5282384165e0f86ef105#check-status-of-issue-import"
}

So far I have found out that it all depends on the contents of foo.json.
The problematic foo.json is available here.
Please help me find out what's wrong with this particular request.

Also, is it true that when using the issue import API I still should not reference
users with @ in the imported tickets to avoid spamming those users?
Or can I safely reference them and don't be afraid that my importing
script will get banned for spamming?

@giorgiosironi
Copy link

It would be great to be able to specify the author of a comment, like we can with the regular API.

As far as I know this isn't possible with the regular Issues and Issue Comments APIs: the author will be the owner of the authentication token being used. The problem in this API is then that the owner of the token will be used for both the issue and all comments, which practically limits you to use a fictional user created for this.

@semiaddict
Copy link

The problem in this API is then that the owner of the token will be used for both the issue and all comments, which practically limits you to use a fictional user created for this.

Has there been any evolution on this ?

@billybooth
Copy link

Just working with the new issues importing API and noticed that it’s not possible to set the author of a comment. All comments are imported using the username of the importer.

It would be great to be able to specify the author of a comment, like we can with the regular API.

Let me go ahead and third or fourth this. Given that the authorized user is an admin for the repo, even if comment authors should be limited to real accounts and not fictionalized, why not require access tokens for any accounts to be posted under and allow the specification of a comment author and token? Or at least allow the contribution of comments with dates for imported issues by additional requests using this API. This limitation basically forces us to utilize the standard Issues API for migrating issues, where we lose our original dates but maintain our team’s commenter/creator identities. With the “import” API, we can preserve dates but we lose our identities. It’s a bit frustrating.

@stmllr
Copy link

stmllr commented Jun 5, 2019

UPDATE: Fixed

It seems the API is currently broken when importing a closed issue.

Importing closed issue: error

echo '{ "issue": { "title": "foo", "body": "bar", "closed": true } }' > issue.json
curl -X POST -H "Authorization: token ${token}" -H "Accept: application/vnd.github.golden-comet-preview+json" -d @issue.json https://api.github.com/repositories/190352707/import/issues
{
  "id": 2453079,
  "status": "pending",
  "url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues/2453079",
  "import_issues_url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues",
  "repository_url": "https://api.github.com/repos/stmllr/test-restored-6625",
  "created_at": "2019-06-05T09:13:24.000Z",
  "updated_at": "2019-06-05T09:13:24.000Z"
}
curl -H GET -H "Authorization: token ${token}" -H "Accept: application/vnd.github.golden-comet-preview+json" https://api.github.com/repositories/190352707/import/issues/2453079
{
  "id": 2453079,
  "status": "failed",
  "url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues/2453079",
  "import_issues_url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues",
  "repository_url": "https://api.github.com/repos/stmllr/test-restored-6625",
  "created_at": "2019-06-05T09:13:24.000Z",
  "updated_at": "2019-06-05T09:13:24.000Z",
  "issue_url": "https://api.github.com/repos/stmllr/test-restored-6625/issues/15",
  "errors": [
    {
      "location": "/issue",
      "resource": "Issue",
      "field": null,
      "value": null,
      "code": "error"
    }
  ]
}

The issue was created in the target repository, but state is open and status check returns "error".
Setting closed_at date did not change the behavior.

Importing open issue

If I set closed to false in the import issue, status check has no errors:

echo '{ "issue": { "title": "foo", "body": "bar", "closed": false } }' > issue.json
curl -X POST -H "Authorization: token ${token}" -H "Accept: application/vnd.github.golden-comet-preview+json" -d @issue.json https://api.github.com/repositories/190352707/import/issues
{
  "id": 2453082,
  "status": "pending",
  "url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues/2453082",
  "import_issues_url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues",
  "repository_url": "https://api.github.com/repos/stmllr/test-restored-6625",
  "created_at": "2019-06-05T09:19:13.000Z",
  "updated_at": "2019-06-05T09:19:13.000Z"
}
curl -H GET -H "Authorization: token ${token}" -H "Accept: application/vnd.github.golden-comet-preview+json" https://api.github.com/repositories/190352707/import/issues/2453082
{
  "id": 2453082,
  "status": "imported",
  "url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues/2453082",
  "import_issues_url": "https://api.github.com/repos/stmllr/test-restored-6625/import/issues",
  "repository_url": "https://api.github.com/repos/stmllr/test-restored-6625",
  "created_at": "2019-06-05T09:19:13.000Z",
  "updated_at": "2019-06-05T09:19:13.000Z",
  "issue_url": "https://api.github.com/repos/stmllr/test-restored-6625/issues/16"
}

@toddr
Copy link

toddr commented Oct 9, 2019

BUG: If you add labels to the import, the time stamp on the event is now. This doesn't reflect last modified luckily but ideally the time stamp would conform to the creation date of the case or something?

@itchyny
Copy link

itchyny commented Nov 6, 2019

Is there a way to import an issue with multiple assignees?

@fmbenhassine
Copy link

@jonmagic thank you for sharing this gist!

Issues and comments created with this API do not trigger any notifications to mentioned users or users watching the repository into which issues are being imported.

Is this still valid? We are planning to use this API to bulk import issues from a legacy issue tracker and we were wondering if watchers of the repo will get notified for each issue created/modified during the process.

Thank you upfront for your feedback.

@jonmagic
Copy link
Author

Please contact [email protected] for latest best practices for imports. Thanks!

@fmbenhassine
Copy link

ok, thank you for your quick reply!

@danielpunkass
Copy link

If anybody else comes across this gist be aware that emails to [email protected] now result in a request to submit requests via the web at https://support.github.com/

@mpdude
Copy link

mpdude commented Feb 3, 2023

Is it possible to control the ID assigned to an imported issue? Or, at least, as a safeguard tell the API that the issue shall not be created if the assigned ID is not a given number that I'd expect?

@jonmagic
Copy link
Author

jonmagic commented Feb 3, 2023

Is it possible to control the ID assigned to an imported issue? Or, at least, as a safeguard tell the API that the issue shall not be created if the assigned ID is not a given number that I'd expect?

Afaik that is not possible and this API was never prioritized to be maintained and updated. As mentioned above submitting a request through https://support.github.com is the best way to seek help/give feedback. Thank you 🙏

@mpdude
Copy link

mpdude commented Feb 3, 2023

I did. Contacted GitHub support, describing the task at hand and they referred me here, with no further information or updates to add.

@jonmagic
Copy link
Author

jonmagic commented Feb 3, 2023

This is an unofficial response from me personally:

We created this API specifically for https://opensource.googleblog.com/2015/03/farewell-to-google-code.html and then left it there for folks to continue using as is. The team that maintained it (my old team) was disbanded during a re-org in early '16 and the API never found a new owner.

I work in a different part of the company now and don't have visibility into it's future but I doubt improvements to this API will ever get prioritized and more likely it will be removed at some point.

Probably not the answer you want but the best I can give based on what I know.

@stmllr
Copy link

stmllr commented Feb 3, 2023

Here's a little "hack" which might work for you:

  1. Find the greatest issue number n from your import issue data.
  2. Create n consecutive issues using the import issue API, use fake issue if you don't need the number, or a "real" issue if it matches one of your desired issue numbers.
  3. Delete the fake issue afterwards.

This will only work once!

@mpdude
Copy link

mpdude commented Feb 4, 2023

💚 to Jonathan for the above statement and 👍 to Steffen for the suggestion.

It's interesting to see that there is an API variant still being exposed (and functional as of today) that allows some more freedom over the date etc. of issues created, with apparently no team and responsibility assigned to it.

@burner
Copy link

burner commented Oct 15, 2023

I have been trying to use this on an organization, after successfully testing this api on a repo created by my test user.

It fails with the result

{"documentation_url":"https:\/\/docs.github.com\/rest","message":"Not Found"}

I don't know how to figure this one out, I triple checked the org and repo name, but no success.

Does anybody have any idea?

@AARP41298
Copy link

@burner do you solve it?
i thought that github-csv-tools was broken, but with the curl example above and Thunder Client i get the same error

@burner
Copy link

burner commented Oct 27, 2023

I re-tried a few days later and it worked. I don't know why. Result can be seen here https://github.com/dlang/visuald/issues

@AARP41298
Copy link

Thanks. I solve it using the GitHub CLI command of
https://docs.github.com/en/rest/issues/issues?apiVersion=2022-11-28#create-an-issue

Maybe curl request for new tokens are disable the first 24hr?

@burner
Copy link

burner commented Dec 16, 2024

I'm happy to announce that we were able to migrate all (open) dlang's bugzilla issues with the api from this gist. Thank you for it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment