Skip to content

Instantly share code, notes, and snippets.

View joshlk's full-sized avatar

Josh Levy-Kramer joshlk

96 followers · 10 following
View GitHub Profile
@joshlk
joshlk / pytorch_backward_ops.ipynb
Created June 21, 2024 09:59
Print PyTorch backwards ops
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@joshlk
joshlk / convolution_matrix_multiplication_equivalence.ipynb
Created April 30, 2024 13:23
Demonstration in PyTorch how convolutions can implemented using matrix multiplication
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@joshlk
joshlk / inference_pretraining_data.ipynb
Last active January 9, 2024 16:47
Inference with pretraining style data for LLM
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@joshlk
joshlk / vocab_sharding.py
Created October 24, 2023 09:29
Vocab sharding using DTensors
from math import ceil
from typing import Optional, Tuple, Union
import torch
from torch import Tensor, nn
from torch.distributed._tensor import DTensor, Replicate, Shard, distribute_module, distribute_tensor
from torch.distributed._tensor.op_schema import OpSchema, OutputSharding
from torch.distributed._tensor.ops.embedding_ops import embedding_rules
from torch.distributed._tensor.ops.utils import register_prop_rule
from torch.distributed._tensor.placement_types import DTensorSpec, _Partial
@joshlk
joshlk / CANNON_MP490_RaspberryPi_CUPS.md
Last active March 24, 2023 13:13
How to add Cannon MP490 printer to RaspberryPi CUPS server
  1. Install Cannon printer drivers sudo apt update; sudo apt install software-properties-common cups-backend-bjnp
  2. Make sure user has password (add password using sudo passwd $user)
  3. Follow instructions to install server https://pimylifeup.com/raspberry-pi-print-server/
  4. Chrome doesn't work to add a printer - worked on Safari
@joshlk
joshlk / stackoverflow_survey.ipynb
Last active January 3, 2023 15:13
Stackoverflow developer survey 2022 analysis of Docker usage between data scientists and devops engineers
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@joshlk
joshlk / conda_total_downloads.ipynb
Last active January 3, 2023 13:24
Obtain total download data for Conda forge
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
@joshlk
joshlk / peak_memory.bash
Last active July 25, 2023 12:49
Measure the peak memory of the system while a command is running (this only includes physical memory and not virtual memory)
#!/usr/bin/env bash
# peak_memory [command ...]
# Run the given command line in the background and kill if script exits
trap 'kill $(jobs -p) 2&> /dev/null' EXIT
"$@" &
pid=$! phys_peak=0 not_avail_peak=0
# while command still running
while ps -p $pid &>/dev/null; do
sleep 1
mem_total="$(cat /proc/meminfo | grep 'MemTotal:' | grep -oe '\([0-9.]*\)')"
@joshlk
joshlk / pandas_excel_formater.py
Last active June 14, 2021 08:38
Format output of excel file written by Pandas. Auto-size, auto-filter, number formats, date-time formats, freeze panels
import xlsxwriter
def excel_formatter(writer, sheet, df, num_format = None, auto_filter = False, auto_fit = False):
workbook = writer.book
worksheet = writer.sheets[sheet]
cols = list(df.columns)
max_row, max_col = df.shape
num_format = num_format if num_format else {}
num_format_types = {
@joshlk
joshlk / confidence_intervals.md
Last active April 21, 2021 09:01
Confidence intervals and quantile regression

Confidence/prediction intervals to quantiles

quantile_higher = (interval_width / 2) + 0.5

Quantile metrics: