Skip to content

Instantly share code, notes, and snippets.

@earthgecko
earthgecko / Pipfile.lock.to.requirements.txt.sh
Created June 29, 2020 12:25
Covert Pipfile.lock to requirements.txt in pure bash
cat Pipfile.lock \
| grep -B1 '"hashes"\|"version": ' \
| grep -v '"markers": \|"hashes": ' \
| grep ": {\|version" \
| sed -e 's/: {$//g' \
| tr '\n' ',' | tr -s ' ' ' ' \
| sed -e 's/, "version": "//g;s/", "/ /g;s/"//g;s/,//g' \
| tr ' ' '\n' \
| grep -v "^$" > requirements.txt
@earthgecko
earthgecko / docker.processes.sh
Created May 20, 2019 14:56
Info about the processes running in docker containers
####
# docker container processes
####
# processes running in docker containers
> /tmp/docker.processes.txt
for i_container_id in $(docker ps | tr -s ' ' ',' | cut -d',' -f1 | grep -v CONTAINER)
do
CONTAINER_NAME=$(docker ps | tr -s ' ' ',' | grep "$i_container_id" | sed -e 's/.*,//g')
echo "# $i_container_id, $CONTAINER_NAME" | tee -a /tmp/docker.processes.txt
@earthgecko
earthgecko / boundary_algorithms.py
Created March 12, 2019 11:32
Allow Boundary to analyse short time series
from __future__ import division
import numpy as np
import scipy
import traceback
import logging
from time import time
from redis import StrictRedis
import sys
import os.path
@earthgecko
earthgecko / extraction.py
Last active September 11, 2018 13:16
Debug tsfresh data_in_chunks list GH418 using groupedby get_group
# -*- coding: utf-8 -*-
# This file as well as the whole tsfresh package are licenced under the MIT licence (see the LICENCE.txt)
# Maximilian Christ (maximilianchrist.com), Blue Yonder Gmbh, 2016
# https://gist.github.com/earthgecko/9e6f2f5c0d48d53ff34284a860a50cde
"""
This module contains the main function to interact with tsfresh: extract features
"""
from __future__ import absolute_import, division
@earthgecko
earthgecko / extraction.py
Last active September 6, 2018 14:22
Debug tsfresh data_in_chunks list GH418
# -*- coding: utf-8 -*-
# This file as well as the whole tsfresh package are licenced under the MIT licence (see the LICENCE.txt)
# Maximilian Christ (maximilianchrist.com), Blue Yonder Gmbh, 2016
# https://gist.github.com/earthgecko/9e6f2f5c0d48d53ff34284a860a50cde
"""
This module contains the main function to interact with tsfresh: extract features
"""
from __future__ import absolute_import, division
@earthgecko
earthgecko / rebrow.redis.password.jwt.token.gist.py
Last active May 31, 2018 17:33
rebrow Redis password token with JWT
# This gist is a basic example of adding Redis password to rebrow in a modified implementation of
# @elky84 https://github.com/marians/rebrow/pull/20 but using PyJWT to encode the password in the POST
# to a JWT token and a client token as a replacement for the plaintext password URL parameter.
# This example includes logging which is not in rebrow and this example rebrow stores the JWT encode string in Redis.
# With normal rebrow this would not be possible and Flask seesion or some internal Python method would need to be
# used. The version of rebrow that is used here is a version that embedded in another application that does have
# access to Redis, hence here rebrow stores the data in Redis, rebrow here requires auth and it is also run behind
# a SSL terminated endpoint and therefore the POST data is encrypted. If rebrow was just run as is, then the POST
# data would not be encrypted and the password would still be sent plaintext.
@earthgecko
earthgecko / etsy.skyline.pands.patch.algorithms.
Last active July 19, 2019 18:42
The required changes to etsy/skyline to get the algorithms.py to run on pandas>=0.18.0
import pandas
import numpy as np
import scipy
import statsmodels.api as sm
import traceback
import logging
from time import time
from msgpack import unpackb, packb
from redis import StrictRedis
@earthgecko
earthgecko / gist:118d168f88ebb37661154e3cb898c1fb
Last active November 2, 2016 19:07
Who's neocortex has seen these before? blue-yonder/tsfresh/i8_add_python3_support
Who's neocortex has seen these before?
These panda scipy py27 py35 are probably buried in some change in pandas relating to some change is Python 3
like range type changing from type list to class or similar. Please just scan through and see if any bells rings.
Perhaps a bell may ring on:
- AttributeError errors 'assertItemsEqual' in a few contexts, but the following may help to pin it down, specifically
- test_mass_quantile
- test_ar_coefficient
@earthgecko
earthgecko / find.change.git.repo
Created November 14, 2015 14:03
Find a change in a git repo via a string in actually git diffs (not just commit messages)
# Find a change via a string in actually git diffs (not just commit messages)
GIT_REPO_DIR="<PATH_TO_YOU_REPO>"
TMP_DIR="/tmp"
STRING_TO_FIND="include openssh_server"
cd "$GIT_REPO_DIR"
> "${TMP_DIR}/git.rev.matches"
git rev-list HEAD |
while read rev; do
@earthgecko
earthgecko / snype_skyline_analyzer.sh
Created February 24, 2015 16:50
skyline analyzer watcher
#!/bin/bash
#### snype_skyline_analyzer.sh ####
#
####
# DESCRIPTION
# This mitigates against analyzer running but it connection timing out
#
# THIS IS JUST A GIST
# THIS IS JUST A GIST