This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
// function to get the client ip address | |
function get_ip() { | |
$ipaddress = ''; | |
if ($_SERVER['HTTP_CLIENT_IP']) | |
$ipaddress = $_SERVER['HTTP_CLIENT_IP']; | |
else if($_SERVER['HTTP_X_FORWARDED_FOR']) | |
$ipaddress = $_SERVER['HTTP_X_FORWARDED_FOR']; | |
else if($_SERVER['HTTP_X_FORWARDED']) | |
$ipaddress = $_SERVER['HTTP_X_FORWARDED']; | |
else if($_SERVER['HTTP_FORWARDED_FOR']) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import os | |
import mutagen | |
import sys | |
# function to find nth occurence of needle in haystack | |
def find_nth(haystack, needle, n): | |
start = haystack.find(needle) | |
while start >= 0 and n > 1: | |
start = haystack.find(needle, start+len(needle)) | |
n -= 1 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
import java.util.Comparator; | |
import java.util.Random; | |
import java.util.Properties; | |
import java.util.Arrays; | |
public class FitnessComparator implements Comparator<double[]>{ | |
@Override | |
public int compare(double[] entry1, double[] entry2) { |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
{ | |
"_brand": "Mammut", | |
"_color_nos": [ | |
"4072", | |
"50134" | |
], | |
"_colors": [ | |
"Olive", | |
"Poseidon" | |
], |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/sh | |
set -e | |
LG=$1 | |
WIKI_DUMP_NAME=${LG}wiki-latest-pages-articles.xml.bz2 | |
WIKI_DUMP_DOWNLOAD_URL=https://dumps.wikimedia.org/${LG}wiki/latest/$WIKI_DUMP_NAME | |
# download latest Wikipedia dump in chosen language | |
echo "Downloading the latest $LG-language Wikipedia dump from $WIKI_DUMP_DOWNLOAD_URL..." | |
wget -c $WIKI_DUMP_DOWNLOAD_URL |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/bin/sh | |
set -e | |
WIKI_DUMP_FILE_IN=$1 | |
WIKI_DUMP_FILE_OUT=${WIKI_DUMP_FILE_IN%%.*}.txt | |
# clone the WikiExtractor repository | |
git clone https://github.com/attardi/wikiextractor.git | |
# extract and clean the chosen Wikipedia dump |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
#!/usr/bin/env python | |
# -*- coding: utf-8 -*- | |
import sys | |
from pathlib import Path | |
from blingfire import text_to_sentences | |
def main(): | |
wiki_dump_file_in = Path(sys.argv[1]) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
implementation | mean execution time | |
---|---|---|
transformers | 6min 42s | |
tokenizers | 45.6s |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
implementation | mean execution time | |
---|---|---|
submit | 1min 8s | |
map | 1min 9s | |
encode_batch | 10.6s |
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
OlderNewer