artforlife

## crawler.md

      
              3 files
            
          
              31 forks
            
          
                6 comments
              
            
              54 stars
            
          
                typehorror
                / crawler.md
            
            
              Last active
              February 19, 2024 03:06
            
              
                Simple Website Crawler (in python)
              
          
    Simple Website Crawler

The following gist is an extract of the article Building a simple crawler. It allows crawling from a URL and for a given number of bounce.
Basic Usage

from crawler import Crawler
crawler = Crawler()
crawler.crawl('http://techcrunch.com/')

displays the urls


## AWS Swarm cluster.md

      
              2 files
            
          
              37 forks
            
          
                1 comment
              
            
              68 stars
            
          
                ghoranyi
                / AWS Swarm cluster.md
            
            
              Last active
              May 31, 2021 05:28
            
              
                Create a Docker 1.12 Swarm cluster on AWS
              
          
    This gist will drive you through creating a Docker 1.12 Swarm cluster (with Swarm mode) on AWS infrastructure.
Prerequisites

You need a few things already prepared in order to get started. You need at least Docker 1.12 set up. I was using the stable version of Docker for mac for preparing this guide.
$ docker --version
Docker version 1.12.0, build 8eab29e

You also need Docker machine installed.