Skip to content

Instantly share code, notes, and snippets.

View ngshiheng's full-sized avatar

Jerry Ng ngshiheng

52 followers · 19 following
View GitHub Profile
@ngshiheng
ngshiheng / cloudSettings
Last active November 25, 2021 04:21
VSCode cloudSettings (Jerry)
{"lastUpload":"2021-11-15T07:51:35.825Z","extensionVersion":"v3.4.3"}
@ngshiheng
ngshiheng / mask_credentials.py
Last active February 27, 2020 00:21
A short Python script to mask all your credentials on your .bashrc file
# Place your secret text file in same directory as this script
# Update your INPUT_FILE & OUTPUT_FILE
import re
INPUT_FILE = "bashrc"
OUTPUT_FILE = "masked_bashrc"
with open(INPUT_FILE) as f:
exports = f.readlines()
@ngshiheng
ngshiheng / pull_request_template.md
Last active December 16, 2019 00:00
PR template

Description

  • Please include a brief summary of the change and which issue is fixed.
  • Please also include relevant motivation and context.
  • List any dependencies that are required for this change.

Fixes # (issue)

Type of change

@ngshiheng
ngshiheng / usefulFunctions.js
Last active March 11, 2020 14:18
Useful JS functions
const sortObjectByKeys = object => {
let sortedObj = {},
keys = Object.keys(object);
keys.sort(function(key1, key2) {
(key1 = key1.toLowerCase()), (key2 = key2.toLowerCase());
if (key1 < key2) return -1;
if (key1 > key2) return 1;
return 0;
});
@ngshiheng
ngshiheng / scrape.py
Created May 2, 2020 15:23
Scrape dynamic content with PyQt5 + bs4
import scrapy
import sys
from bs4 import BeautifulSoup
from PyQt5.QtWebEngineWidgets import QWebEnginePage
from PyQt5.QtWidgets import QApplication
from PyQt5.QtCore import QUrl
class Client(QWebEnginePage):
def __init__(self, url):
@ngshiheng
ngshiheng / filters.py
Last active February 7, 2022 05:53
JSONField Models in Graphene Django https://medium.com/open-graphql/jsonfield-models-in-graphene-django-308ae43d14ee
from django_filters import FilterSet, OrderingFilter
from .models import Person
class PersonFilter(FilterSet):
class Meta:
model = Person
fields = ['first_name', 'last_name']
@ngshiheng
ngshiheng / example.ini
Last active February 7, 2022 05:53
How to Package Python Selenium Applications With PyInstaller https://jerrynsh.com/how-to-package-python-selenium-applications-with-pyinstaller/
[chromedriver]
path = driver\chromedriver.exe
[website]
url = http://quotes.toscrape.com/
[delay]
seconds = 1
@ngshiheng
ngshiheng / article_types.py
Last active February 7, 2022 05:54
Solving N+1 Problem with Dataloader in Python Graphene Django https://jerrynsh.com/solving-n-1-in-graphql-python-with-dataloader/
class ArticleNode(DjangoObjectType):
dataloader_reporter = Field('starter.types.ReporterNode', description='Get a single Reporter detail using dataloader.')
class Meta:
model = Article
interfaces = (Node,)
filterset_class = ArticleFilter
connection_class = CountableConnectionBase
@staticmethod
@ngshiheng
ngshiheng / graphene_django_starter.json
Last active February 7, 2022 05:51
Django Graphene Starter https://github.com/ngshiheng/django-graphene-starter
{
"_type": "export",
"__export_format": 4,
"__export_date": "2021-03-13T05:05:13.722Z",
"__export_source": "insomnia.desktop.app:v2021.1.1",
"resources": [
{
"_id": "req_0c9319edd46b4a5b92e2bc2fd65bf2ec",
"parentId": "wrk_0a110636708841cc8f82e73c43c2585b",
"modified": 1615391416735,
@ngshiheng
ngshiheng / pipelines.py
Last active October 11, 2022 04:34
An example pipeline that uses bulk insert while saving scrapped item into database using SQLAlchemy.
import logging
from scrapy import Spider
from sqlalchemy.orm import sessionmaker
from example.items import ProductItem
from example.models import Price, Product, create_table, db_connect
logger = logging.getLogger(__name__)