kachayev

"""
Imagine we have a training loop for an agent. E.g. PPO, or DQN, or whatever.

What is the easiest way to convert this into a selfplay? 

To make this happen we want to run 2 identical loop: 1 loop for each agent.
But. There are 2 sync points: `env.reset()`, and each `env.step()`.
Is there a way to avoid duplicating code for setting/loading agent model, creating/updating
buffers, running training loops etc... Yes, it is. By utilizing generators protocol: `yield` and `send`.

kachayev

from functools import reduce
from itertools import compress
from operator import mul

def iterate(f, initial):
  """Keeps applying function `f` to accumulated state, yielding each intermediate result.
  
  E.g. iterate(lambda k: k*k, 2) -> 2,4,16,64,256,65536,..."""
  state = initial
  while True:

kachayev

#
# The following is the simplification of EncoderRNN code from
# "NLP From Scratch: Translation with a Sequence to Sequence Network and Attention"
# PyTorch tutorial (link: https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html)
#
# In fact, `nn.GRU` module can execute loop mechanics over the given input sequence when provided
# with such. In the tutorial, the sentence is presented as a tensor of 1-word sequences hence the
# loop is handled in `train` and `evaluation` "manually". Given the fact the `nn.Embedding` handles
# set of indecies and `nn.GRU` executes loops when given (seq_len, batch_size, elem_dim) shaped
# input, encoder could be constructed as the following:

kachayev

Same data structure that HPACK is using right now:

Benchmark                                                      Mode  Cnt         Score        Error  Units
QpackStaticTableBenchmark.findFieldNameAndValueMatch          thrpt    5   1829339.334 ± 121376.837  ops/s
QpackStaticTableBenchmark.findFieldNameRefForEmptyField       thrpt    5   2341547.888 ±  27061.256  ops/s
QpackStaticTableBenchmark.findFieldNameRefForMultipleMatches  thrpt    5   3637243.304 ±  74366.646  ops/s
QpackStaticTableBenchmark.findFieldNameRefForSingleMatch      thrpt    5   6284579.831 ± 247083.622  ops/s
QpackStaticTableBenchmark.findFieldNotFound                   thrpt    5  67002791.898 ± 372832.308  ops/s

kachayev

Aleph, Part I

A lot has happened these last few weeks. The key focus was on implementing new features, so let's enumerate all of them.

WebSockets

• #422 "Handle WebSocket handshake timeouts".

Initially implemented by Denis Shilov. Thanks a lot! I've reimplemented the functionality partially to avoid usage of Manifold's timer. Mostly becase of performance considerations. I've also opened the same issue for Netty as this functionality is not covered by built-in websocket protocol handlers. And it was already implemented here by Qin Shicheng.

kachayev

(require '[manifold.deferred :as d])

(defn long-boring-computation []
  (let [result (d/deferred)]
    (d/future
      (dotimes [_ 100]
        (when-not (d/realized? result)
          ;; block the thread is still a bad idea :(
          (Thread/sleep 1e3)))
      (d/success! result "I'm done with this"))

kachayev

Aleph, Async, HTTP, Clojure

I've been working with Aleph rougly for last 5 years, actively contributing to the library for last 2 (or so). I also put some effort into spreading the word about it, including educational tech talks, like "Deep HTTP Dive Throught Aleph & Netty". But the more I talk to people the more confusion I find, mostly about how Aleph works and what can you expect when adding it to your stack. Clojurists Together has recently announced Aleph to get Q1 funding, I think it's a good time to share my priorities and thoughts on development plans that were mentioned in the blog post. Hope the community would find it interesting and helpful.

Aleph describes itself as "asynchronous communication for Clojure" library. And you should probably pay a good portion of your attention to the first word: "asynchronous".

kachayev

# -*- coding: utf-8 -*-

from tabulate import tabulate

class Var(object):
    def __init__(self, name):
        self.name = name
        self.value = None

    def bind(self, value):

kachayev

(defn fetch-user
  [id]
  (fetch-item id))

(defn fetch-post
  [id]
  (fetch-item id))

(defn timeline-ids
  [username]

kachayev

Channels Are Not Enough

... or Why Pipelining Is Not That Easy

Golang Concurrency Patterns for brave and smart.

By @kachayev

Intro