kzk

日本Hadoopユーザー会は、第4回目のカンファレンス「Hadoop Conference 
Japan 2013 Winter」を1月21日に東京ビッグサイトにて開催します。

> http://hcj2013w.eventbrite.com/

オリジナル開発者 Doug Cuttingからのメッセージ、LINE, Treasure Data,
Amazon Elastic MapReduce等のキーノートを始め、Apache Hadoopに関す
る技術や事例を一同に集め、多くの方が出会える場として企画を進めています。

早くも席が埋まり始めているので、なるべくお早めにご登録下さい。

kzk

#!/usr/lib/fluent/ruby/bin/ruby                                                                                                   
ENV["FLUENT_CONF"]="/etc/td-agent/td-agent.conf"
ENV["FLUENT_PLUGIN"]="/etc/td-agent/plugin"
ENV["FLUENT_SOCKET"]="/var/run/td-agent/td-agent.sock"

# Please add these lines
ENV["GEM_HOME"]="/usr/lib/fluent/ruby/lib/ruby/gems/1.9.1/"
ENV["GEM_PATH"]="/usr/lib/fluent/ruby/lib/ruby/gems/1.9.1/"

load "/usr/lib/fluent/ruby/bin/fluentd"

kzk

$ gem install fluent-plugin-mongo      
Fetching: bson-1.7.0.gem (100%)
Fetching: mongo-1.7.0.gem (100%)
Fetching: fluent-plugin-mongo-0.6.7.gem (100%)
Successfully installed bson-1.7.0
Successfully installed mongo-1.7.0
Successfully installed fluent-plugin-mongo-0.6.7
3 gems installed
Installing ri documentation for bson-1.7.0...
Installing ri documentation for mongo-1.7.0...

kzk

module UnicornWorkerKiller
  def self.kill_self(logger, start_time)
    alive_sec = (Time.now - start_time).to_i

    i = 0
    while true
      i += 1
      sig = :TERM
      if i > 10     # TODO configurable TERM MAX
        sig = :QUIT

kzk

$ bundle exec je ./script/rails s

kzk

gem 'jemalloc'

kzk

from fluent import sender
from fluent import event
sender.setup('td.test_db', host='localhost', port=24224)
event.Event('follow', {
    'from': 'userA',
    'to':   'userB'
    })

kzk

# from https://gist.github.com/1258681
# # your config.ru
# require 'unicorn_killer'
# use UnicornKiller::MaxRequests, 1000
# use UnicornKiller::Oom, 400 * 1024

module UnicornKiller
  module Kill
    def quit
      sec = (Time.now - @process_start).to_i

kzk

# Unicorn self-process killer
require ::File.expand_path('../lib/unicorn/unicorn_killer',  __FILE__)
use UnicornKiller::MaxRequests, 10240 + Random.rand(10240)
use UnicornKiller::Oom, 96 * 1024 + Random.rand(32) * 1024

# Out-Of-Band GC
require 'unicorn/oob_gc'
use Unicorn::OobGC

kzk

# What the timeout for killing busy workers is, in seconds
timeout 60

# Whether the app should be pre-loaded
preload_app true

# How many worker processes
worker_processes 4

# before/after forks