Skip to content

Instantly share code, notes, and snippets.

@jlewi

jlewi/Airflow

Created January 10, 2018 05:43
Show Gist options
  • Save jlewi/61d830d3e1c4487cf4d6a18b75eb941a to your computer and use it in GitHub Desktop.
Save jlewi/61d830d3e1c4487cf4d6a18b75eb941a to your computer and use it in GitHub Desktop.
Download ZIP
Airflow logs
Raw
Airflow
Airflow
DAGs
Data Profiling
Browse
Admin
Docs
About
05:43 UTC
OnOff DAG: tf_k8s_tests schedule: None
Graph View
Tree View
Task Duration
Task Tries
Landing Times
Gantt
Details
Code
Refresh
Task Instance: build_images
2018-01-10 03:03:24
Task Instance Details
Rendered Template
Log
XCom
Log by attempts
1
2
*** Reading local log.
[2018-01-10 03:03:40,937] {cli.py:379} INFO - Running on host airflow-2497854871-0dr9k
[2018-01-10 03:03:40,992] {models.py:1173} INFO - Dependencies all met for <TaskInstance: tf_k8s_tests.build_images 2018-01-10 03:03:24 [queued]>
[2018-01-10 03:03:41,016] {models.py:1173} INFO - Dependencies all met for <TaskInstance: tf_k8s_tests.build_images 2018-01-10 03:03:24 [queued]>
[2018-01-10 03:03:41,017] {models.py:1383} INFO -
--------------------------------------------------------------------------------
Starting attempt 1 of 2
--------------------------------------------------------------------------------
[2018-01-10 03:03:41,060] {models.py:1404} INFO - Executing <Task(PythonOperator): build_images> on 2018-01-10 03:03:24
[2018-01-10 03:03:41,061] {base_task_runner.py:115} INFO - Running: ['bash', '-c', u'airflow run tf_k8s_tests build_images 2018-01-10T03:03:24 --job_id 3342 --raw -sd DAGS_FOLDER/e2e_tests_dag.py']
[2018-01-10 03:03:43,734] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:43,733] {__init__.py:45} INFO - Using executor LocalExecutor
[2018-01-10 03:03:43,884] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:43,883] {models.py:189} INFO - Filling up the DagBag from /usr/local/airflow/dags/e2e_tests_dag.py
[2018-01-10 03:03:43,888] {base_task_runner.py:98} INFO - Subtask: /usr/local/lib/python2.7/site-packages/apache_airflow-1.9.0rc2-py2.7.egg/airflow/utils/helpers.py:351: DeprecationWarning: Importing PythonOperator directly from <module 'airflow.operators' from '/usr/local/lib/python2.7/site-packages/apache_airflow-1.9.0rc2-py2.7.egg/airflow/operators/__init__.pyc'> has been deprecated. Please import from '<module 'airflow.operators' from '/usr/local/lib/python2.7/site-packages/apache_airflow-1.9.0rc2-py2.7.egg/airflow/operators/__init__.pyc'>.[operator_module]' instead. Support for direct imports will be dropped entirely in Airflow 2.0.
[2018-01-10 03:03:43,888] {base_task_runner.py:98} INFO - Subtask: DeprecationWarning)
[2018-01-10 03:03:44,629] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,628] {e2e_tests_dag.py:176} INFO - dag_id: tf_k8s_tests
[2018-01-10 03:03:44,630] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,629] {e2e_tests_dag.py:177} INFO - run_id: 2018-01-10T03:03:24
[2018-01-10 03:03:44,642] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,642] {e2e_tests_dag.py:180} INFO - Using run_dir=/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24
[2018-01-10 03:03:44,667] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,666] {e2e_tests_dag.py:183} INFO - Using src_dir=/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s
[2018-01-10 03:03:44,668] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,667] {e2e_tests_dag.py:186} INFO - gcs_path gs://mlkube-testing-airflow/runs/tf_k8s_tests/2018-01-10T03_03_24
[2018-01-10 03:03:44,668] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,667] {e2e_tests_dag.py:191} INFO - conf={u'PULL_NUMBER': u'273', u'PULL_BASE_SHA': u'9c93d5fe62e1b727039ac3ecb634488653cd1a48', u'ARTIFACTS_PATH': u'gs://kubernetes-jenkins/pr-logs/pull/tensorflow_k8s/273/tf-k8s-presubmit/334/artifacts', u'PULL_PULL_SHA': u'14e4ac25a6b1dc1ce957602ed63ccc51374f2e90'}
[2018-01-10 03:03:44,669] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,667] {e2e_tests_dag.py:193} INFO - artifacts_path gs://kubernetes-jenkins/pr-logs/pull/tensorflow_k8s/273/tf-k8s-presubmit/334/artifacts
[2018-01-10 03:03:44,670] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,669] {e2e_tests_dag.py:83} INFO - Removing /opt/tensorflow_k8s from PYTHONPATH
[2018-01-10 03:03:44,694] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,693] {e2e_tests_dag.py:104} INFO - Using util located at /opt/tensorflow_k8s/py/util.pyc
[2018-01-10 03:03:44,695] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,694] {util.py:36} INFO - Running: python -m py.release build --src_dir=/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s --build_info_path=gs://mlkube-testing-airflow/runs/tf_k8s_tests/2018-01-10T03_03_24/build_info.yaml --releases_path=gs://mlkube-testing-airflow/runs/tf_k8s_tests/2018-01-10T03_03_24 --project=mlkube-testing
[2018-01-10 03:03:44,695] {base_task_runner.py:98} INFO - Subtask: cwd=None
[2018-01-10 03:03:44,696] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:03:44,694] {util.py:46} INFO - Running: Environment:
[2018-01-10 03:03:44,696] {base_task_runner.py:98} INFO - Subtask: AIRFLOW_PORT=tcp://10.63.240.90:80
[2018-01-10 03:03:44,696] {base_task_runner.py:98} INFO - Subtask: AIRFLOW_PORT_80_TCP=tcp://10.63.240.90:80
[2018-01-10 03:03:44,697] {base_task_runner.py:98} INFO - Subtask: AIRFLOW_PORT_80_TCP_ADDR=10.63.240.90
[2018-01-10 03:03:44,697] {base_task_runner.py:98} INFO - Subtask: AIRFLOW_PORT_80_TCP_PORT=80
[2018-01-10 03:03:44,697] {base_task_runner.py:98} INFO - Subtask: AIRFLOW_PORT_80_TCP_PROTO=tcp
[2018-01-10 03:03:44,698] {base_task_runner.py:98} INFO - Subtask: AIRFLOW_SERVICE_HOST=10.63.240.90
[2018-01-10 03:03:44,698] {base_task_runner.py:98} INFO - Subtask: AIRFLOW_SERVICE_PORT=80
[2018-01-10 03:03:44,699] {base_task_runner.py:98} INFO - Subtask: CLOUDSDK_CORE_DISABLE_PROMPTS=1
[2018-01-10 03:03:44,699] {base_task_runner.py:98} INFO - Subtask: DEBIAN_FRONTEND=noninteractive
[2018-01-10 03:03:44,699] {base_task_runner.py:98} INFO - Subtask: DECK_PORT=tcp://10.63.247.198:80
[2018-01-10 03:03:44,700] {base_task_runner.py:98} INFO - Subtask: DECK_PORT_80_TCP=tcp://10.63.247.198:80
[2018-01-10 03:03:44,700] {base_task_runner.py:98} INFO - Subtask: DECK_PORT_80_TCP_ADDR=10.63.247.198
[2018-01-10 03:03:44,700] {base_task_runner.py:98} INFO - Subtask: DECK_PORT_80_TCP_PORT=80
[2018-01-10 03:03:44,701] {base_task_runner.py:98} INFO - Subtask: DECK_PORT_80_TCP_PROTO=tcp
[2018-01-10 03:03:44,701] {base_task_runner.py:98} INFO - Subtask: DECK_SERVICE_HOST=10.63.247.198
[2018-01-10 03:03:44,702] {base_task_runner.py:98} INFO - Subtask: DECK_SERVICE_PORT=80
[2018-01-10 03:03:44,702] {base_task_runner.py:98} INFO - Subtask: GOOGLE_APPLICATION_CREDENTIALS=/var/secrets/google/key.json
[2018-01-10 03:03:44,704] {base_task_runner.py:98} INFO - Subtask: GOPATH=/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/go
[2018-01-10 03:03:44,705] {base_task_runner.py:98} INFO - Subtask: GPG_KEY=C01E1CAD5EA2C4F0B8E3571504C367C218ADD4FF
[2018-01-10 03:03:44,705] {base_task_runner.py:98} INFO - Subtask: HOME=/usr/local/airflow
[2018-01-10 03:03:44,706] {base_task_runner.py:98} INFO - Subtask: HOOK_PORT=tcp://10.63.245.64:8888
[2018-01-10 03:03:44,706] {base_task_runner.py:98} INFO - Subtask: HOOK_PORT_8888_TCP=tcp://10.63.245.64:8888
[2018-01-10 03:03:44,707] {base_task_runner.py:98} INFO - Subtask: HOOK_PORT_8888_TCP_ADDR=10.63.245.64
[2018-01-10 03:03:44,707] {base_task_runner.py:98} INFO - Subtask: HOOK_PORT_8888_TCP_PORT=8888
[2018-01-10 03:03:44,707] {base_task_runner.py:98} INFO - Subtask: HOOK_PORT_8888_TCP_PROTO=tcp
[2018-01-10 03:03:44,708] {base_task_runner.py:98} INFO - Subtask: HOOK_SERVICE_HOST=10.63.245.64
[2018-01-10 03:03:44,708] {base_task_runner.py:98} INFO - Subtask: HOOK_SERVICE_PORT=8888
[2018-01-10 03:03:44,714] {base_task_runner.py:98} INFO - Subtask: HOSTNAME=airflow-2497854871-0dr9k
[2018-01-10 03:03:44,714] {base_task_runner.py:98} INFO - Subtask: KUBERNETES_PORT=tcp://10.63.240.1:443
[2018-01-10 03:03:44,714] {base_task_runner.py:98} INFO - Subtask: KUBERNETES_PORT_443_TCP=tcp://10.63.240.1:443
[2018-01-10 03:03:44,715] {base_task_runner.py:98} INFO - Subtask: KUBERNETES_PORT_443_TCP_ADDR=10.63.240.1
[2018-01-10 03:03:44,715] {base_task_runner.py:98} INFO - Subtask: KUBERNETES_PORT_443_TCP_PORT=443
[2018-01-10 03:03:44,716] {base_task_runner.py:98} INFO - Subtask: KUBERNETES_PORT_443_TCP_PROTO=tcp
[2018-01-10 03:03:44,716] {base_task_runner.py:98} INFO - Subtask: KUBERNETES_SERVICE_HOST=10.63.240.1
[2018-01-10 03:03:44,716] {base_task_runner.py:98} INFO - Subtask: KUBERNETES_SERVICE_PORT=443
[2018-01-10 03:03:44,717] {base_task_runner.py:98} INFO - Subtask: KUBERNETES_SERVICE_PORT_HTTPS=443
[2018-01-10 03:03:44,717] {base_task_runner.py:98} INFO - Subtask: LANG=en_US.UTF-8
[2018-01-10 03:03:44,717] {base_task_runner.py:98} INFO - Subtask: LANGUAGE=en_US.UTF-8
[2018-01-10 03:03:44,718] {base_task_runner.py:98} INFO - Subtask: LC_ALL=en_US.UTF-8
[2018-01-10 03:03:44,718] {base_task_runner.py:98} INFO - Subtask: LC_CTYPE=en_US.UTF-8
[2018-01-10 03:03:44,718] {base_task_runner.py:98} INFO - Subtask: LC_MESSAGES=en_US.UTF-8
[2018-01-10 03:03:44,719] {base_task_runner.py:98} INFO - Subtask: PATH=/google-cloud-sdk/bin:/workspace:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/go/bin
[2018-01-10 03:03:44,719] {base_task_runner.py:98} INFO - Subtask: POSTGRES_HOST=localhost
[2018-01-10 03:03:44,720] {base_task_runner.py:98} INFO - Subtask: PWD=/usr/local/airflow
[2018-01-10 03:03:44,720] {base_task_runner.py:98} INFO - Subtask: PYTHONPATH=:/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s
[2018-01-10 03:03:44,720] {base_task_runner.py:98} INFO - Subtask: PYTHON_PIP_VERSION=9.0.1
[2018-01-10 03:03:44,722] {base_task_runner.py:98} INFO - Subtask: PYTHON_VERSION=2.7.14
[2018-01-10 03:03:44,723] {base_task_runner.py:98} INFO - Subtask: SHLVL=2
[2018-01-10 03:03:44,723] {base_task_runner.py:98} INFO - Subtask: SRC_DIR=/var/lib/data/runs
[2018-01-10 03:03:44,724] {base_task_runner.py:98} INFO - Subtask: TERM=linux
[2018-01-10 03:03:44,725] {base_task_runner.py:98} INFO - Subtask: _=/usr/local/bin/airflow
[2018-01-10 03:05:41,663] {base_task_runner.py:98} INFO - Subtask: [2018-01-10 03:05:41,662] {util.py:74} INFO - Subprocess output:
[2018-01-10 03:05:41,663] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:03:46|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py|320| Use --src_dir=/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s
[2018-01-10 03:05:41,664] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:03:46|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py|329| /var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/go/src/github.com/tensorflow/k8s does not exist.
[2018-01-10 03:05:41,664] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:03:46|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py|336| Creating symbolic link /var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/go/src/github.com/tensorflow/k8s pointing to /var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s
[2018-01-10 03:05:41,665] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:03:46|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py|359| vendor directory exists; not installing go dependencies.
[2018-01-10 03:05:41,665] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:03:46|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py|126| context_dir: /tmp/tmpTfJobCrdContextbXeAs4
[2018-01-10 03:05:41,665] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:03:47|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|35| Running: go install -ldflags -X github.com/tensorflow/k8s/version.GitSHA=14e4ac2 github.com/tensorflow/k8s/cmd/tf_operator
[2018-01-10 03:05:41,666] {base_task_runner.py:98} INFO - Subtask: cwd=None
[2018-01-10 03:05:41,666] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:04:35|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|64| Subprocess output:
[2018-01-10 03:05:41,667] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,667] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:04:35|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|35| Running: go install github.com/tensorflow/k8s/cmd/tf_operator
[2018-01-10 03:05:41,667] {base_task_runner.py:98} INFO - Subtask: cwd=None
[2018-01-10 03:05:41,668] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:04:35|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|64| Subprocess output:
[2018-01-10 03:05:41,668] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,668] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:04:35|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|35| Running: go install github.com/tensorflow/k8s/test/e2e
[2018-01-10 03:05:41,669] {base_task_runner.py:98} INFO - Subtask: cwd=None
[2018-01-10 03:05:41,669] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:04:39|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|64| Subprocess output:
[2018-01-10 03:05:41,669] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,670] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:04:39|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|35| Running: go install github.com/tensorflow/k8s/dashboard/backend
[2018-01-10 03:05:41,670] {base_task_runner.py:98} INFO - Subtask: cwd=None
[2018-01-10 03:05:41,675] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:04:45|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|64| Subprocess output:
[2018-01-10 03:05:41,675] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,676] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:04:45|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|35| Running: yarn --cwd /var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/dashboard/frontend install
[2018-01-10 03:05:41,676] {base_task_runner.py:98} INFO - Subtask: cwd=None
[2018-01-10 03:05:41,676] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:05:06|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|64| Subprocess output:
[2018-01-10 03:05:41,677] {base_task_runner.py:98} INFO - Subtask: yarn install v1.3.2
[2018-01-10 03:05:41,677] {base_task_runner.py:98} INFO - Subtask: [1/4] Resolving packages...
[2018-01-10 03:05:41,677] {base_task_runner.py:98} INFO - Subtask: [2/4] Fetching packages...
[2018-01-10 03:05:41,678] {base_task_runner.py:98} INFO - Subtask: info [email protected]: The platform "linux" is incompatible with this module.
[2018-01-10 03:05:41,678] {base_task_runner.py:98} INFO - Subtask: info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[2018-01-10 03:05:41,679] {base_task_runner.py:98} INFO - Subtask: info [email protected]: The platform "linux" is incompatible with this module.
[2018-01-10 03:05:41,679] {base_task_runner.py:98} INFO - Subtask: info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[2018-01-10 03:05:41,679] {base_task_runner.py:98} INFO - Subtask: [3/4] Linking dependencies...
[2018-01-10 03:05:41,680] {base_task_runner.py:98} INFO - Subtask: [4/4] Building fresh packages...
[2018-01-10 03:05:41,680] {base_task_runner.py:98} INFO - Subtask: Done in 19.98s.
[2018-01-10 03:05:41,680] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,681] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:05:06|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|35| Running: yarn --cwd /var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/dashboard/frontend build
[2018-01-10 03:05:41,681] {base_task_runner.py:98} INFO - Subtask: cwd=None
[2018-01-10 03:05:41,681] {base_task_runner.py:98} INFO - Subtask: INFO|2018-01-10T03:05:41|/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/util.py|64| Subprocess output:
[2018-01-10 03:05:41,682] {base_task_runner.py:98} INFO - Subtask: yarn run v1.3.2
[2018-01-10 03:05:41,682] {base_task_runner.py:98} INFO - Subtask: $ react-scripts build
[2018-01-10 03:05:41,682] {base_task_runner.py:98} INFO - Subtask: Creating an optimized production build...
[2018-01-10 03:05:41,683] {base_task_runner.py:98} INFO - Subtask: Compiled successfully.
[2018-01-10 03:05:41,683] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,683] {base_task_runner.py:98} INFO - Subtask: File sizes after gzip:
[2018-01-10 03:05:41,684] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,684] {base_task_runner.py:98} INFO - Subtask: 127.05 KB build/static/js/main.7154f589.js
[2018-01-10 03:05:41,684] {base_task_runner.py:98} INFO - Subtask: 137 B build/static/css/main.e1b6790e.css
[2018-01-10 03:05:41,685] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,685] {base_task_runner.py:98} INFO - Subtask: The project was built assuming it is hosted at the server root.
[2018-01-10 03:05:41,686] {base_task_runner.py:98} INFO - Subtask: To override this, specify the homepage in your package.json.
[2018-01-10 03:05:41,686] {base_task_runner.py:98} INFO - Subtask: For example, add this to build it for GitHub Pages:
[2018-01-10 03:05:41,686] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,687] {base_task_runner.py:98} INFO - Subtask: "homepage" : "http://myname.github.io/myapp",
[2018-01-10 03:05:41,687] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,687] {base_task_runner.py:98} INFO - Subtask: The build folder is ready to be deployed.
[2018-01-10 03:05:41,695] {base_task_runner.py:98} INFO - Subtask: You may serve it with a static server:
[2018-01-10 03:05:41,695] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,696] {base_task_runner.py:98} INFO - Subtask: yarn global add serve
[2018-01-10 03:05:41,696] {base_task_runner.py:98} INFO - Subtask: serve -s build
[2018-01-10 03:05:41,696] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,697] {base_task_runner.py:98} INFO - Subtask: Done in 34.61s.
[2018-01-10 03:05:41,697] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,697] {base_task_runner.py:98} INFO - Subtask: Traceback (most recent call last):
[2018-01-10 03:05:41,698] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/runpy.py", line 174, in _run_module_as_main
[2018-01-10 03:05:41,698] {base_task_runner.py:98} INFO - Subtask: "__main__", fname, loader, pkg_name)
[2018-01-10 03:05:41,698] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code
[2018-01-10 03:05:41,699] {base_task_runner.py:98} INFO - Subtask: exec code in run_globals
[2018-01-10 03:05:41,699] {base_task_runner.py:98} INFO - Subtask: File "/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py", line 682, in <module>
[2018-01-10 03:05:41,699] {base_task_runner.py:98} INFO - Subtask: main()
[2018-01-10 03:05:41,700] {base_task_runner.py:98} INFO - Subtask: File "/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py", line 679, in main
[2018-01-10 03:05:41,700] {base_task_runner.py:98} INFO - Subtask: args.func(args)
[2018-01-10 03:05:41,700] {base_task_runner.py:98} INFO - Subtask: File "/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py", line 363, in build
[2018-01-10 03:05:41,701] {base_task_runner.py:98} INFO - Subtask: build_and_push(go_dir, args.src_dir, args)
[2018-01-10 03:05:41,701] {base_task_runner.py:98} INFO - Subtask: File "/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py", line 382, in build_and_push
[2018-01-10 03:05:41,702] {base_task_runner.py:98} INFO - Subtask: build_info_path=args.build_info_path)
[2018-01-10 03:05:41,702] {base_task_runner.py:98} INFO - Subtask: File "/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py", line 228, in build_and_push_artifacts
[2018-01-10 03:05:41,702] {base_task_runner.py:98} INFO - Subtask: build_info = build_operator_image(src_dir, registry, project=gcb_project)
[2018-01-10 03:05:41,703] {base_task_runner.py:98} INFO - Subtask: File "/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/py/release.py", line 168, in build_operator_image
[2018-01-10 03:05:41,703] {base_task_runner.py:98} INFO - Subtask: shutil.copyfile(src_path, dest_path)
[2018-01-10 03:05:41,703] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/shutil.py", line 96, in copyfile
[2018-01-10 03:05:41,704] {base_task_runner.py:98} INFO - Subtask: with open(src, 'rb') as fsrc:
[2018-01-10 03:05:41,704] {base_task_runner.py:98} INFO - Subtask: IOError: [Errno 2] No such file or directory: '/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s/images/tf_operator/Dockerfile'
[2018-01-10 03:05:41,704] {base_task_runner.py:98} INFO - Subtask:
[2018-01-10 03:05:41,710] {base_task_runner.py:98} INFO - Subtask: Traceback (most recent call last):
[2018-01-10 03:05:41,710] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/bin/airflow", line 4, in <module>
[2018-01-10 03:05:41,711] {base_task_runner.py:98} INFO - Subtask: __import__('pkg_resources').run_script('apache-airflow==1.9.0rc2', 'airflow')
[2018-01-10 03:05:41,711] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 750, in run_script
[2018-01-10 03:05:41,712] {base_task_runner.py:98} INFO - Subtask: self.require(requires)[0].run_script(script_name, ns)
[2018-01-10 03:05:41,712] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1527, in run_script
[2018-01-10 03:05:41,712] {base_task_runner.py:98} INFO - Subtask: exec(code, namespace, namespace)
[2018-01-10 03:05:41,713] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/site-packages/apache_airflow-1.9.0rc2-py2.7.egg/EGG-INFO/scripts/airflow", line 27, in <module>
[2018-01-10 03:05:41,713] {base_task_runner.py:98} INFO - Subtask: args.func(args)
[2018-01-10 03:05:41,713] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/site-packages/apache_airflow-1.9.0rc2-py2.7.egg/airflow/bin/cli.py", line 397, in run
[2018-01-10 03:05:41,714] {base_task_runner.py:98} INFO - Subtask: pool=args.pool,
[2018-01-10 03:05:41,714] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/site-packages/apache_airflow-1.9.0rc2-py2.7.egg/airflow/utils/db.py", line 50, in wrapper
[2018-01-10 03:05:41,715] {base_task_runner.py:98} INFO - Subtask: result = func(*args, **kwargs)
[2018-01-10 03:05:41,715] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/site-packages/apache_airflow-1.9.0rc2-py2.7.egg/airflow/models.py", line 1469, in _run_raw_task
[2018-01-10 03:05:41,715] {base_task_runner.py:98} INFO - Subtask: result = task_copy.execute(context=context)
[2018-01-10 03:05:41,716] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/site-packages/apache_airflow-1.9.0rc2-py2.7.egg/airflow/operators/python_operator.py", line 89, in execute
[2018-01-10 03:05:41,716] {base_task_runner.py:98} INFO - Subtask: return_value = self.execute_callable()
[2018-01-10 03:05:41,716] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/site-packages/apache_airflow-1.9.0rc2-py2.7.egg/airflow/operators/python_operator.py", line 94, in execute_callable
[2018-01-10 03:05:41,717] {base_task_runner.py:98} INFO - Subtask: return self.python_callable(*self.op_args, **self.op_kwargs)
[2018-01-10 03:05:41,717] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/airflow/dags/e2e_tests_dag.py", line 211, in build_images
[2018-01-10 03:05:41,717] {base_task_runner.py:98} INFO - Subtask: run(ti, args, use_print=True, dryrun=dryrun, env=newenv)
[2018-01-10 03:05:41,718] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/airflow/dags/e2e_tests_dag.py", line 105, in run
[2018-01-10 03:05:41,718] {base_task_runner.py:98} INFO - Subtask: util.run(*extra_args, **kwargs)
[2018-01-10 03:05:41,718] {base_task_runner.py:98} INFO - Subtask: File "/opt/tensorflow_k8s/py/util.py", line 58, in run
[2018-01-10 03:05:41,719] {base_task_runner.py:98} INFO - Subtask: stderr=subprocess.STDOUT).decode("utf-8")
[2018-01-10 03:05:41,719] {base_task_runner.py:98} INFO - Subtask: File "/usr/local/lib/python2.7/subprocess.py", line 219, in check_output
[2018-01-10 03:05:41,719] {base_task_runner.py:98} INFO - Subtask: raise CalledProcessError(retcode, cmd, output=output)
[2018-01-10 03:05:41,720] {base_task_runner.py:98} INFO - Subtask: subprocess.CalledProcessError: Command '['python', '-m', 'py.release', 'build', u'--src_dir=/var/lib/data/runs/tf_k8s_tests/2018-01-10T03_03_24/tensorflow_k8s', u'--build_info_path=gs://mlkube-testing-airflow/runs/tf_k8s_tests/2018-01-10T03_03_24/build_info.yaml', u'--releases_path=gs://mlkube-testing-airflow/runs/tf_k8s_tests/2018-01-10T03_03_24', '--project=mlkube-testing']' returned non-zero exit status 1
@jlewi
Copy link
Author

jlewi commented Jan 10, 2018

I gave you access to the project GCP mlkube-testing so if you want you can access the Airflow on the Kubernetes cluster where its running.
But its probably easier to debug it just by running

python -m py.release build

@ScorpioCPH
Copy link

@jlewi Thanks so much!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment