zhangw

Common

export OPT=/opt
export BUILDS=/some/where/mini_linux
mkdir -p $BUILDS

Linux kernel

zhangw

# 启动测试
import openai
openai.api_key = "" # Not support yet
openai.api_base = "http://127.0.0.1:18888/v1"

# 选择你启动的模型
model = "codellama_34b"

# create a chat completion
completion = openai.ChatCompletion.create(

zhangw

python -m fastchat.serve.cli --gptq-wbits 4 --gptq-group 64 --model-path $HOME/.cache/modelscope/hub/codefuse-ai/CodeFuse-CodeLlama-34B-4bits --device cuda --style rich

python -m fastchat.serve.controller
python -m fastchat.serve.model_worker --gptq-wbits 4 --gptq-group 64 --model-path $HOME/.cache/modelscope/hub/codefuse-ai/CodeFuse-CodeLlama-34B-4bits --device cuda
python -m fastchat.serve.test_message --model-name CodeFuse-CodeLlama-34B-4bits
python -m fastchat.serve.gradio_web_server

zhangw

import os
import torch
import time
from modelscope import AutoTokenizer, snapshot_download
from auto_gptq import AutoGPTQForCausalLM

os.environ["TOKENIZERS_PARALLELISM"] = "false"

def load_model_tokenizer(model_path):
    """

zhangw

Libuv and libev, two I/O libraries with similar names, recently had the privilege to use both libraries to write something. Now let's talk about my own subjective expression of common and different points.

The topic of high-performance network programming has been discussed. Asynchronous, asynchronous, or asynchronous. Whether it is epoll or kqueue, it is always indispensable to the asynchronous topic.

Libuv is asynchronous, and libev is synchronous multiplexing IO multiplexing.

Libev is a simple encapsulation of system I/O reuse. Basically, it solves the problem of different APIs between epoll and kqueuq. Ensure that programs written using livev's API can run on most *nix platforms. However, the disadvantages of libev are also obvious. Because it basically just encapsulates the Event Library, it is inconvenient to use. For example, accept(3) requires manual setnonblocking after connection. EAGAIN, EWOULDBLOCK, and EINTER need to be detected when reading from a socket. This is a

zhangw

• Scaling k8s daemonset down to zero

kubectl -n kube-system patch daemonset myDaemonset -p '{"spec": {"template": {"spec": {"nodeSelector": {"non-existing": "true"}}}}}'

• Scaling up k8s daemonset

kubectl -n kube-system patch daemonset myDaemonset --type json -p='[{"op": "remove", "path": "/spec/template/spec/nodeSelector/non-existing"}]'

zhangw

Attach to a running process with

  gdb -x trace-dyn-mem -p $PID

After every malloc the returned value (the allocated address) will be read from the RAX (64 bits) register.

After every free the last item in the backtrace (the free itself) will be shown. With the libc6-dbg package installed you can see the address passed as the first argument of free.

zhangw

# check the specific thrift version from fe-core, for 2.5.3
ag THRIFT_DOWNLOAD thirdparty/vars.sh
## 90:THRIFT_DOWNLOAD="http://archive.apache.org/dist/thrift/0.13.0/thrift-0.13.0.tar.gz"

# install the specific thrift version, using brew which help to compile automatically
brew tap-new $USER/local-tap
brew extract --version='0.13.0' thrift $USER/local-tap
brew install [email protected]
## check the installation path, for me
/opt/homebrew/Cellar/[email protected]/0.13.0/bin/thrift --version

zhangw

## install musl
sudo pacman -Syu musl

## install headers of musl
sudo pacman -Syu kernel-headers-musl

## cargo add musl target and toolchain
rustup target add --toolchain stable x86_64-unknown-linux-musl

## download openssl source code, version is 3.0.7

zhangw

# Install the toolchain
```bash
brew tap SergioBenitez/osxct
brew install x86_64-unknown-linux-gnu
```

# this should get installed in:
# /opt/homebrew/Cellar/x86_64-unknown-linux-gnu/

# Installing the macOS OpenSSL - if not already installed