5 entertaining things you can find with the GitHub Search API

00-about-search-api-examples.md

5 entertaining things you can find with the GitHub Search API

Let's have some command-line fun with curl, jq, and the new GitHub Search API.

Today we're looking for:

• The hottest repositories created in the last week.
• The oldest user accounts with zero followers. (So sad.)
• Issue trends by language.
• Clojure projects defying convention.
• John Resig's hobbies outside of JavaScript.

01-trending-repos.md

Find the hottest repositories created in the last week

Keep an eye on the latest trending repos.

# We'll use the `date` command to get the date for "7 days ago"
$ date -v-7d '+%Y-%m-%d'
# => 2013-07-15

$ curl -G https://api.github.com/search/repositories       \
    --data-urlencode "q=created:>`date -v-7d '+%Y-%m-%d'`" \
    --data-urlencode "sort=stars"                          \
    --data-urlencode "order=desc"                          \
    -H "Accept: application/vnd.github.preview"            \
    | jq ".items[0,1,2] | {name, description, language, watchers_count, html_url}"

{
  "html_url": "https://github.com/mame/quine-relay",
  "watchers_count": 2492,
  "language": "Ruby",
  "description": "An uroboros program with 50 programming languages",
  "name": "quine-relay"
}
{
  "html_url": "https://github.com/jehna/VerbalExpressions",
  "watchers_count": 483,
  "language": "JavaScript",
  "description": "JavaScript Regular expressions made easy",
  "name": "VerbalExpressions"
}
{
  "html_url": "https://github.com/applidium/ADTransitionController",
  "watchers_count": 340,
  "language": "Objective-C",
  "description": "UINavigationController with custom transitions",
  "name": "ADTransitionController"
}

Check out the Repository Search API docs for more details.

02-lonely-users.md

Find the oldest user accounts with zero followers

Go follow these poor souls.

$ curl -G https://api.github.com/search/users   \
    --data-urlencode 'q=followers:0'            \
    --data-urlencode 'sort=joined'              \
    --data-urlencode 'order=asc'                \
    -H 'Accept: application/vnd.github.preview' \
    | jq '.items[0,1,2] | {html_url, login, id}'

{
  "id": 30,
  "login": "fanvsfan",
  "html_url": "https://github.com/fanvsfan"
}
{
  "id": 32,
  "login": "railsjitsu",
  "html_url": "https://github.com/railsjitsu"
}
{
  "id": 44,
  "login": "errfree",
  "html_url": "https://github.com/errfree"
}

Check out the User Search API docs for more details.

03-issue-trends.md

How many issues do Ruby developers create each day?

Track issue trends over time.

for i in {1..7}
do
    created_on=`date -v-"${i}"d '+%Y-%m-%d'`
    issue_count=$(                                                      \
        curl -G https://api.github.com/search/issues                    \
        --data-urlencode "q=language:ruby created:$created_on"          \
        -H "Authorization: token REDACTED"                              \
        -H "Accept: application/vnd.github.preview" | jq ".total_count" \
    )

    echo "$created_on: $issue_count"
done

2013-07-22: 1174
2013-07-21: 716
2013-07-20: 687
2013-07-19: 1336
2013-07-18: 1348
2013-07-17: 1471
2013-07-16: 1386

Check out the Issue Search API docs for more details.

04-rebel-clojure-projects.md

Find recently-indexed Clojure projects using an MIT license

Rebels! I bet these dudes don't like hammocks either.

$ curl -G https://api.github.com/search/code          \
    --data-urlencode 'q=MIT License path:project.clj' \
    --data-urlencode 'sort=indexed'                   \
    --data-urlencode 'order=desc'                     \
    -H 'Accept: application/vnd.github.preview'       \
    | jq '.items[0,1,2] | {description: (.repository.description), name: (.repository.full_name), html_url}'

{
  "html_url": "https://github.com/royvandewater/optparse/blob/e4bb8558405ffd7ba6a14718b89e7cd418b5565e/project.clj",
  "name": "royvandewater/optparse",
  "description": "Option parser for clojure"
}
{
  "html_url": "https://github.com/SnootyMonkey/coming-soon/blob/108decfd74338ba5428580c8cd156f684484d3b2/project.clj",
  "name": "SnootyMonkey/coming-soon",
  "description": "coming-soon is a simple Clojure/ClojureScript/Redis 'landing page' application that takes just a few minute to setup"
}
{
  "html_url": "https://github.com/rreas/ring-test/blob/5fd78404eec4f81033ae073c5a42ea8a55ccc75a/project.clj",
  "name": "rreas/ring-test",
  "description": "An integration test framework for ring web applications."
}

Check out the Code Search API docs for more details.

05-resig.md

What does John Resig work on other than JavaScript?

$ curl -G https://api.github.com/search/repositories   \
    --data-urlencode 'q=@jeresig -language:javascript' \
    -H 'Accept: application/vnd.github.preview'        \
    | jq '.items[] | {html_url, watchers_count, language, name}'

{
  "name": "processing-js",
  "language": "Java",
  "watchers_count": 1455,
  "html_url": "https://github.com/jeresig/processing-js"
}
{
  "name": "selectortest",
  "language": null,
  "watchers_count": 12,
  "html_url": "https://github.com/jeresig/selectortest"
}
{
  "name": "wtpa-bot",
  "language": "Perl",
  "watchers_count": 9,
  "html_url": "https://github.com/jeresig/wtpa-bot"
}
{
  "name": "jeresig.github.com",
  "language": null,
  "watchers_count": 6,
  "html_url": "https://github.com/jeresig/jeresig.github.com"
}
{
  "name": "apples2artworks",
  "language": "Python",
  "watchers_count": 1,
  "html_url": "https://github.com/jeresig/apples2artworks"
}
{
  "name": "datacook",
  "language": "Perl",
  "watchers_count": 0,
  "html_url": "https://github.com/jeresig/datacook"
}

@jeresig has a repo with zero people watching it? This changes my whole worldview.

---

Check out the Repository Search API docs for more details.

06-trolls.md

Find the most commented issues in the past month

OK. OK. You were only expecting 5 examples. But since you read this far, here's a bonus one.

# We'll use the `date` command to get the date for "1 month ago"
$ date -v-1m '+%Y-%m-%d'
# => 2013-06-23

$ curl -G https://api.github.com/search/issues             \
    --data-urlencode "q=created:>`date -v-7d '+%Y-%m-%d'`" \
    --data-urlencode 'sort=comments'                       \
    --data-urlencode 'order=desc'                          \
    -H 'Accept: application/vnd.github.preview'            \
    | jq '.items[0,1,2] | {html_url, title, comments}'

I ran this a couple months ago, when we were first sketching out the search API. What did I find? A Pull Request to remove all press representatives from bitcoin.org. Followed by another Pull Request to counteract the first one.

Interesting times.

{
  "comments": 158,
  "title": "Remove press representatives",
  "html_url": "https://github.com/bitcoin/bitcoin.org/issues/152"
}
{
  "comments": 151,
  "title": "Add several independent voices to the Press Center page",
  "html_url": "https://github.com/bitcoin/bitcoin.org/issues/162"
}
{
  "comments": 150,
  "title": "Select and implement a template engine (or The issue to discuss everything Framework refactoring)",
  "html_url": "https://github.com/joomla/jissues/issues/86"
}

Check out the Issue Search API docs for more details.

The date command on Ubuntu Linux does not have the '-v' flag.

Try @rochefort's git-trend. It's a command line utility that shows you trending github repos. Another is my github-trending gem that fetches daily, weekly and monthly trending repositories on github.

Does any of the current github python api does advanced search like this?

Its probably only entertaining on MacOS date: invalid option -- 'v'

created_on=`date --date="${i} days ago" '+%Y-%m-%d'

and add --silent to curl argument

How about Find the hottest repositories created in the last week, just like https://github.com/trending

Your 04-rebel-clojure-projects.md query doesn't work. I wonder if they added this requirement after you posted this.

  "message": "Validation Failed",
  "errors": [
    {
      "message": "Must include at least one user, organization, or repository",
      "resource": "Search",
      "field": "q",
      "code": "invalid"
    }
  ],
  "documentation_url": "https://developer.github.com/v3/search/"
}

@nealmcb This has to do with authentication. Quoting from the "Considerations for code search" section of the article on searching code:

Logged in users can search all public repositories while anonymous searches must include a limit on org:, user:, or repo:.

If you add authentication to your calls, things should work as described...

Jason, basically I want to find the pull requests of a repository which are older than 7 days of the present date so my query for getting the pull requests is as follows:::

https://api.github.com/repos/octokit/octokit.rb/pulls

the response is as follows
[
{
"url": "https://api.github.com/repos/octokit/octokit.rb/pulls/951",
"id": 147991930,
"html_url": "https://github.com/octokit/octokit.rb/pull/951",
"diff_url": "https://github.com/octokit/octokit.rb/pull/951.diff",
"patch_url": "https://github.com/octokit/octokit.rb/pull/951.patch",
"issue_url": "https://api.github.com/repos/octokit/octokit.rb/issues/951",
"number": 951,
"state": "open",
"locked": false,
"title": "CI against Ruby 2.2.8, 2.3.5, 2.4.2",
"user": {
"login": "yatmsu",
"id": 436515,
"avatar_url": "https://avatars1.githubusercontent.com/u/436515?v=4",
"gravatar_id": "",
"url": "https://api.github.com/users/yatmsu",
"html_url": "https://github.com/yatmsu",
"followers_url": "https://api.github.com/users/yatmsu/followers",
"following_url": "https://api.github.com/users/yatmsu/following{/other_user}",
"gists_url": "https://api.github.com/users/yatmsu/gists{/gist_id}",
"starred_url": "https://api.github.com/users/yatmsu/starred{/owner}{/repo}",
"subscriptions_url": "https://api.github.com/users/yatmsu/subscriptions",
"organizations_url": "https://api.github.com/users/yatmsu/orgs",
"repos_url": "https://api.github.com/users/yatmsu/repos",
"events_url": "https://api.github.com/users/yatmsu/events{/privacy}",
"received_events_url": "https://api.github.com/users/yatmsu/received_events",
"type": "User",
"site_admin": false
},
"body": "https://www.ruby-lang.org/en/news/2017/09/14/ruby-2-2-8-released/\r\nhttps://www.ruby-lang.org/en/news/2017/09/14/ruby-2-3-5-released/\r\nhttps://www.ruby-lang.org/en/news/2017/09/14/ruby-2-4-2-released/",
"created_at": "2017-10-22T12:51:14Z",
"updated_at": "2017-10-22T13:36:43Z",
"closed_at": null,
"merged_at": null,
"merge_commit_sha": "d0ea0ed9a32999b62c154515c4ebb351486243ef",
"assignee": null,
"assignees": [],
"requested_reviewers": [],
"milestone": null,
"commits_url": "https://api.github.com/repos/octokit/octokit.rb/pulls/951/commits",
"review_comments_url": "https://api.github.com/repos/octokit/octokit.rb/pulls/951/comments",
"review_comment_url": "https://api.github.com/repos/octokit/octokit.rb/pulls/comments{/number}",
"comments_url": "https://api.github.com/repos/octokit/octokit.rb/issues/951/comments",
"statuses_url": "https://api.github.com/repos/octokit/octokit.rb/statuses/d1dc7dd2e8e3c21a24f4cfe2a7d8d785557f0b54",
"head": {

so what parameters should I add after pulls to get the pull requests based on date (older than 7 days),
can you give me the exact query, (I can see the created at parameter in the response but I don't know how to use it)

I want to use this get get call in my python script, so I think we can't use --data--urlencode.
can you give me the exact query so that it works in postman and my python script.

thanks

Find all github.io repositories that are actually online?

Is Code search API useful for searching, filename:x across all repositories with "xyz in the file content? As of now, it throws me error saying that it, need User/repo/Organization

The above requests seem not to be working anymore?