Created
January 13, 2023 16:42
-
-
Save jfgrimm/08e90b7e6fd73ef93230a10efe4c38bf to your computer and use it in GitHub Desktop.
(partial) EasyBuild log for failed build of /users/jfg508/TF_logging/TensorFlow-2.9.1-foss-2022a-CUDA-11.7.0.eb (easyblock PR(s) #2854)
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| PyEval_EvalCode | |
| _PyRun_SimpleFileObject | |
| _PyRun_AnyFileObject | |
| Py_RunMain | |
| Py_BytesMain | |
| __libc_start_main | |
| *** End stack trace *** | |
| 2023-01-13 15:47:23.921889: I tensorflow/core/platform/status.cc:102] Generated non-OK status: "INTERNAL: Uninitialized stream". *** Begin stack trace *** | |
| tensorflow::CurrentStackTrace[abi:cxx11]() | |
| tensorflow::Status::Status(tensorflow::error::Code, absl::lts_20211102::string_view, std::vector<tensorflow::StackFrame, std::allocator<tensorflow::StackFrame> >&&) | |
| stream_executor::Stream::Stream(stream_executor::StreamExecutor*) | |
| xla::StreamPool::BorrowStream(stream_executor::StreamExecutor*) | |
| xla::Backend::BorrowStream(stream_executor::StreamExecutor*) | |
| xla::Backend::BorrowStream(int) | |
| xla::LocalExecutable::RunHelper(absl::lts_20211102::Span<xla::Shape const* const>, xla::ExecutableRunOptions) | |
| tensorflow::StatusOr<xla::ExecutionOutput> xla::LocalExecutable::AsyncCallAndBlockHostUntilDone<xla::ExecutionOutput>(absl::lts_20211102::Span<xla::Shape const* const>, xla::ExecutableRunOptions const&, std::function<tensorflow::StatusOr<xla::ExecutionOutput> (xla::ExecutableRunOptions const&)>) | |
| xla::LocalExecutable::Run(std::vector<xla::ExecutionInput, std::allocator<xla::ExecutionInput> >, xla::ExecutableRunOptions) | |
| tensorflow::XlaLocalLaunchBase::Compute(tensorflow::OpKernelContext*) | |
| tensorflow::ThreadPoolDevice::Compute(tensorflow::OpKernel*, tensorflow::OpKernelContext*) | |
| tensorflow::KernelAndDeviceOp::Run(tensorflow::ScopedStepContainer*, tensorflow::EagerKernelArgs const&, std::vector<absl::lts_20211102::variant<tensorflow::Tensor, tensorflow::TensorShape>, std::allocator<absl::lts_20211102::variant<tensorflow::Tensor, tensorflow::TensorShape> > >*, tensorflow::CancellationManager*, absl::lts_20211102::optional<tensorflow::EagerFunctionParams> const&, absl::lts_20211102::optional<tensorflow::ManagedStackTrace> const&, tensorflow::CoordinationServiceAgent*) | |
| tensorflow::EagerKernelExecute(tensorflow::EagerContext*, absl::lts_20211102::InlinedVector<tensorflow::TensorHandle*, 4ul, std::allocator<tensorflow::TensorHandle*> > const&, absl::lts_20211102::optional<tensorflow::EagerFunctionParams> const&, std::unique_ptr<tensorflow::KernelAndDevice, tensorflow::core::RefCountDeleter> const&, tensorflow::GraphCollector*, tensorflow::CancellationManager*, absl::lts_20211102::Span<tensorflow::TensorHandle*>, absl::lts_20211102::optional<tensorflow::ManagedStackTrace> const&) | |
| tensorflow::ExecuteNode::Run() | |
| tensorflow::EagerExecutor::SyncExecute(tensorflow::EagerNode*) | |
| tensorflow::EagerExecute(tensorflow::EagerOperation*, tensorflow::TensorHandle**, int*) | |
| tensorflow::EagerOperation::Execute(absl::lts_20211102::Span<tensorflow::AbstractTensorHandle*>, int*) | |
| tensorflow::CustomDeviceOpHandler::Execute(tensorflow::ImmediateExecutionOperation*, tensorflow::ImmediateExecutionTensorHandle**, int*) | |
| TFE_Execute | |
| TFE_Py_ExecuteCancelable(TFE_Context*, char const*, char const*, absl::lts_20211102::InlinedVector<TFE_TensorHandle*, 4ul, std::allocator<TFE_TensorHandle*> >*, _object*, TFE_CancellationManager*, absl::lts_20211102::InlinedVector<TFE_TensorHandle*, 2ul, std::allocator<TFE_TensorHandle*> >*, TF_Status*) | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| PyEval_EvalCode | |
| _PyRun_SimpleFileObject | |
| _PyRun_AnyFileObject | |
| Py_RunMain | |
| Py_BytesMain | |
| __libc_start_main | |
| *** End stack trace *** | |
| 2023-01-13 15:47:23.921944: F tensorflow/stream_executor/stream.cc:213] Check failed: VLOG_IS_ON(1) | |
| Fatal Python error: Aborted | |
| Current thread 0x00007f0d3cab4740 (most recent call first): | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/eager/execute.py", line 54 in quick_execute | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/eager/function.py", line 497 in call | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/eager/function.py", line 1860 in _call_flat | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/eager/def_function.py", line 986 in _call | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/eager/def_function.py", line 915 in __call__ | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/util/traceback_utils.py", line 141 in error_handler | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/framework/constant_op_test.py", line 84 in test_eager_const_xla | |
| File "/opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/unittest/case.py", line 549 in _callTestMethod | |
| File "/opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/unittest/case.py", line 591 in run | |
| File "/opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/unittest/case.py", line 650 in __call__ | |
| File "/opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/unittest/suite.py", line 122 in run | |
| File "/opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/unittest/suite.py", line 84 in __call__ | |
| File "/opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/unittest/suite.py", line 122 in run | |
| File "/opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/unittest/suite.py", line 84 in __call__ | |
| File "/opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/unittest/runner.py", line 184 in run | |
| File "/opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/unittest/main.py", line 271 in runTests | |
| File "/opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/python3.10/unittest/main.py", line 101 in __init__ | |
| File "/users/jfg508/scratch/easybuild-dev/software/TensorFlow/2.9.1-foss-2022a-CUDA-11.7.0/lib/python3.10/site-packages/absl/testing/absltest.py", line 2518 in _run_and_get_tests_result | |
| File "/users/jfg508/scratch/easybuild-dev/software/TensorFlow/2.9.1-foss-2022a-CUDA-11.7.0/lib/python3.10/site-packages/absl/testing/absltest.py", line 2549 in run_tests | |
| File "/users/jfg508/scratch/easybuild-dev/software/TensorFlow/2.9.1-foss-2022a-CUDA-11.7.0/lib/python3.10/site-packages/absl/testing/absltest.py", line 2146 in _run_in_app | |
| File "/users/jfg508/scratch/easybuild-dev/software/TensorFlow/2.9.1-foss-2022a-CUDA-11.7.0/lib/python3.10/site-packages/absl/testing/absltest.py", line 2051 in main | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/platform/googletest.py", line 51 in g_main | |
| File "/users/jfg508/scratch/easybuild-dev/software/TensorFlow/2.9.1-foss-2022a-CUDA-11.7.0/lib/python3.10/site-packages/absl/app.py", line 254 in _run_main | |
| File "/users/jfg508/scratch/easybuild-dev/software/TensorFlow/2.9.1-foss-2022a-CUDA-11.7.0/lib/python3.10/site-packages/absl/app.py", line 308 in run | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/platform/googletest.py", line 60 in main_wrapper | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/platform/benchmark.py", line 503 in benchmarks_main | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/platform/googletest.py", line 62 in main | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/platform/test.py", line 56 in main | |
| File "/dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/framework/constant_op_test.py", line 134 in <module> | |
| Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, tensorflow.python.framework.fast_tensor_util (total: 14) | |
| *** Received signal 6 *** | |
| *** BEGIN MANGLED STACK TRACE *** | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/../../_solib_local/_U_S_Stensorflow_Spython_C_Upywrap_Uparallel_Udevice.so___Utensorflow/libtensorflow_framework.so.2(+0xec82bc)[0x7f0d325542bc] | |
| /lib64/libpthread.so.0(+0xf630)[0x7f0d3bbb0630] | |
| /lib64/libpthread.so.0(raise+0x2b)[0x7f0d3bbb04fb] | |
| /lib64/libpthread.so.0(+0xf630)[0x7f0d3bbb0630] | |
| /lib64/libc.so.6(gsignal+0x37)[0x7f0d3b606387] | |
| /lib64/libc.so.6(abort+0x148)[0x7f0d3b607a78] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/../../_solib_local/_U_S_Stensorflow_Spython_C_Upywrap_Uparallel_Udevice.so___Utensorflow/libtensorflow_framework.so.2(+0x12e1aa7)[0x7f0d3296daa7] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/../../_solib_local/_U_S_Stensorflow_Spython_C_Upywrap_Uparallel_Udevice.so___Utensorflow/libtensorflow_framework.so.2(+0x58ffff)[0x7f0d31c1bfff] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/../../_solib_local/_U_S_Stensorflow_Spython_C_Upywrap_Uparallel_Udevice.so___Utensorflow/libtensorflow_framework.so.2(_ZN15stream_executor6StreamC2EPNS_14StreamExecutorE+0x31c)[0x7f0d32739f1c] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN3xla10StreamPool12BorrowStreamEPN15stream_executor14StreamExecutorE+0x221)[0x7f0d23bda831] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN3xla7Backend12BorrowStreamEPN15stream_executor14StreamExecutorE+0x15b)[0x7f0d224e855b] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN3xla7Backend12BorrowStreamEi+0x3b)[0x7f0d224e874b] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN3xla15LocalExecutable9RunHelperEN4absl12lts_202111024SpanIKPKNS_5ShapeEEENS_20ExecutableRunOptionsE+0x65a)[0x7f0d224b253a] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN3xla15LocalExecutable30AsyncCallAndBlockHostUntilDoneINS_15ExecutionOutputEEEN10tensorflow8StatusOrIT_EEN4absl12lts_202111024SpanIKPKNS_5ShapeEEERKNS_20ExecutableRunOptionsESt8functionIFS6_SH_EE+0x72)[0x7f0d224b60f2] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN3xla15LocalExecutable3RunESt6vectorINS_14ExecutionInputESaIS2_EENS_20ExecutableRunOptionsE+0x169)[0x7f0d224b6b89] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(+0x523f69c)[0x7f0d1a51d69c] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow18XlaLocalLaunchBase7ComputeEPNS_15OpKernelContextE+0x802)[0x7f0d1a51e3c2] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/../../_solib_local/_U_S_Stensorflow_Spython_C_Upywrap_Uparallel_Udevice.so___Utensorflow/libtensorflow_framework.so.2(_ZN10tensorflow16ThreadPoolDevice7ComputeEPNS_8OpKernelEPNS_15OpKernelContextE+0x4b)[0x7f0d320608ab] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow17KernelAndDeviceOp3RunEPNS_19ScopedStepContainerERKNS_15EagerKernelArgsEPSt6vectorIN4absl12lts_202111027variantIJNS_6TensorENS_11TensorShapeEEEESaISC_EEPNS_19CancellationManagerERKNS8_8optionalINS_19EagerFunctionParamsEEERKNSI_INS_17ManagedStackTraceEEEPNS_24CoordinationServiceAgentE+0x9cd)[0x7f0d2192de1d] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow18EagerKernelExecuteEPNS_12EagerContextERKN4absl12lts_2021110213InlinedVectorIPNS_12TensorHandleELm4ESaIS6_EEERKNS3_8optionalINS_19EagerFunctionParamsEEERKSt10unique_ptrINS_15KernelAndDeviceENS_4core15RefCountDeleterEEPNS_14GraphCollectorEPNS_19CancellationManagerENS3_4SpanIS6_EERKNSB_INS_17ManagedStackTraceEEE+0x232)[0x7f0d1a773282] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow11ExecuteNode3RunEv+0x229)[0x7f0d1a7741b9] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow13EagerExecutor11SyncExecuteEPNS_9EagerNodeE+0x4e2)[0x7f0d21e92082] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(+0x5492950)[0x7f0d1a770950] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow12EagerExecuteEPNS_14EagerOperationEPPNS_12TensorHandleEPi+0x27b)[0x7f0d1a770e4b] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow14EagerOperation7ExecuteEN4absl12lts_202111024SpanIPNS_20AbstractTensorHandleEEEPi+0xbd)[0x7f0d1a51421d] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_ZN10tensorflow21CustomDeviceOpHandler7ExecuteEPNS_27ImmediateExecutionOperationEPPNS_30ImmediateExecutionTensorHandleEPi+0x499)[0x7f0d21938989] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(TFE_Execute+0x63)[0x7f0d19da29b3] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tensorflow_internal.so(_Z24TFE_Py_ExecuteCancelableP11TFE_ContextPKcS2_PN4absl12lts_2021110213InlinedVectorIP16TFE_TensorHandleLm4ESaIS7_EEEP7_objectP23TFE_CancellationManagerPNS5_IS7_Lm2ES8_EEP9TF_Status+0x34d)[0x7f0d19a84c2d] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tfe.so(+0x8d018)[0x7f0bcce3c018] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tfe.so(+0x9c977)[0x7f0bcce4b977] | |
| /dev/shm/jfg508/TensorFlow/2.9.1/foss-2022a-CUDA-11.7.0/TensorFlow/bazel-root/2f58f7c5a0571d5816161c526b354498/execroot/org_tensorflow/bazel-out/k8-opt/bin/tensorflow/python/framework/constant_op_test.runfiles/org_tensorflow/tensorflow/python/_pywrap_tfe.so(+0x8f062)[0x7f0bcce3e062] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x14223e)[0x7f0d3c63c23e] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_MakeTpCall+0x74)[0x7f0d3c639634] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x594b)[0x7f0d3c6353db] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x1b2e)[0x7f0d3c6315be] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x148ec1)[0x7f0d3c642ec1] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x1b2e)[0x7f0d3c6315be] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x746)[0x7f0d3c6301d6] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x148f8a)[0x7f0d3c642f8a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x3762)[0x7f0d3c6331f2] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x3762)[0x7f0d3c6331f2] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_FastCallDictTstate+0x8d)[0x7f0d3c638f3d] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_Call_Prepend+0x4a)[0x7f0d3c64178a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x1e4060)[0x7f0d3c6de060] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_MakeTpCall+0x74)[0x7f0d3c639634] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x21fa)[0x7f0d3c631c8a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x148ec1)[0x7f0d3c642ec1] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x347)[0x7f0d3c62fdd7] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x746)[0x7f0d3c6301d6] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x148f8a)[0x7f0d3c642f8a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x3762)[0x7f0d3c6331f2] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_FastCallDictTstate+0x8d)[0x7f0d3c638f3d] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_Call_Prepend+0x4a)[0x7f0d3c64178a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x1e4060)[0x7f0d3c6de060] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_MakeTpCall+0x74)[0x7f0d3c639634] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x21fa)[0x7f0d3c631c8a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x148f8a)[0x7f0d3c642f8a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x3762)[0x7f0d3c6331f2] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_FastCallDictTstate+0x8d)[0x7f0d3c638f3d] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_Call_Prepend+0x4a)[0x7f0d3c64178a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x1e4060)[0x7f0d3c6de060] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_MakeTpCall+0x74)[0x7f0d3c639634] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x21fa)[0x7f0d3c631c8a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x148f8a)[0x7f0d3c642f8a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x3762)[0x7f0d3c6331f2] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_FastCallDictTstate+0x8d)[0x7f0d3c638f3d] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_Call_Prepend+0x4a)[0x7f0d3c64178a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x1e4060)[0x7f0d3c6de060] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_MakeTpCall+0x74)[0x7f0d3c639634] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x21fa)[0x7f0d3c631c8a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x746)[0x7f0d3c6301d6] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x746)[0x7f0d3c6301d6] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_FastCallDictTstate+0xe9)[0x7f0d3c638f99] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x14749f)[0x7f0d3c64149f] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x13f88b)[0x7f0d3c63988b] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyObject_Call+0x6f)[0x7f0d3c64330f] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x3762)[0x7f0d3c6331f2] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x347)[0x7f0d3c62fdd7] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x347)[0x7f0d3c62fdd7] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x347)[0x7f0d3c62fdd7] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x1b2e)[0x7f0d3c6315be] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x347)[0x7f0d3c62fdd7] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x347)[0x7f0d3c62fdd7] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x1b2e)[0x7f0d3c6315be] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x347)[0x7f0d3c62fdd7] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x1b2e)[0x7f0d3c6315be] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x11a6)[0x7f0d3c630c36] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyEval_EvalFrameDefault+0x11a6)[0x7f0d3c630c36] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x134bfb)[0x7f0d3c62ebfb] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(PyEval_EvalCode+0x87)[0x7f0d3c6a5047] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x1bc664)[0x7f0d3c6b6664] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0x1b8136)[0x7f0d3c6b2136] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(+0xac3f9)[0x7f0d3c5a63f9] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyRun_SimpleFileObject+0x380)[0x7f0d3c5a5b9a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(_PyRun_AnyFileObject+0x84)[0x7f0d3c5a7862] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(Py_RunMain+0x3aa)[0x7f0d3c6c285a] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/lib/libpython3.10.so.1.0(Py_BytesMain+0x29)[0x7f0d3c697a59] | |
| /lib64/libc.so.6(__libc_start_main+0xf5)[0x7f0d3b5f2555] | |
| /opt/apps/easybuild/software/Python/3.10.4-GCCcore-11.3.0/bin/python[0x401089] | |
| *** END MANGLED STACK TRACE *** | |
| *** Begin stack trace *** | |
| tensorflow::CurrentStackTrace[abi:cxx11]() | |
| raise | |
| gsignal | |
| abort | |
| stream_executor::Stream::Stream(stream_executor::StreamExecutor*) | |
| xla::StreamPool::BorrowStream(stream_executor::StreamExecutor*) | |
| xla::Backend::BorrowStream(stream_executor::StreamExecutor*) | |
| xla::Backend::BorrowStream(int) | |
| xla::LocalExecutable::RunHelper(absl::lts_20211102::Span<xla::Shape const* const>, xla::ExecutableRunOptions) | |
| tensorflow::StatusOr<xla::ExecutionOutput> xla::LocalExecutable::AsyncCallAndBlockHostUntilDone<xla::ExecutionOutput>(absl::lts_20211102::Span<xla::Shape const* const>, xla::ExecutableRunOptions const&, std::function<tensorflow::StatusOr<xla::ExecutionOutput> (xla::ExecutableRunOptions const&)>) | |
| xla::LocalExecutable::Run(std::vector<xla::ExecutionInput, std::allocator<xla::ExecutionInput> >, xla::ExecutableRunOptions) | |
| tensorflow::XlaLocalLaunchBase::Compute(tensorflow::OpKernelContext*) | |
| tensorflow::ThreadPoolDevice::Compute(tensorflow::OpKernel*, tensorflow::OpKernelContext*) | |
| tensorflow::KernelAndDeviceOp::Run(tensorflow::ScopedStepContainer*, tensorflow::EagerKernelArgs const&, std::vector<absl::lts_20211102::variant<tensorflow::Tensor, tensorflow::TensorShape>, std::allocator<absl::lts_20211102::variant<tensorflow::Tensor, tensorflow::TensorShape> > >*, tensorflow::CancellationManager*, absl::lts_20211102::optional<tensorflow::EagerFunctionParams> const&, absl::lts_20211102::optional<tensorflow::ManagedStackTrace> const&, tensorflow::CoordinationServiceAgent*) | |
| tensorflow::EagerKernelExecute(tensorflow::EagerContext*, absl::lts_20211102::InlinedVector<tensorflow::TensorHandle*, 4ul, std::allocator<tensorflow::TensorHandle*> > const&, absl::lts_20211102::optional<tensorflow::EagerFunctionParams> const&, std::unique_ptr<tensorflow::KernelAndDevice, tensorflow::core::RefCountDeleter> const&, tensorflow::GraphCollector*, tensorflow::CancellationManager*, absl::lts_20211102::Span<tensorflow::TensorHandle*>, absl::lts_20211102::optional<tensorflow::ManagedStackTrace> const&) | |
| tensorflow::ExecuteNode::Run() | |
| tensorflow::EagerExecutor::SyncExecute(tensorflow::EagerNode*) | |
| tensorflow::EagerExecute(tensorflow::EagerOperation*, tensorflow::TensorHandle**, int*) | |
| tensorflow::EagerOperation::Execute(absl::lts_20211102::Span<tensorflow::AbstractTensorHandle*>, int*) | |
| tensorflow::CustomDeviceOpHandler::Execute(tensorflow::ImmediateExecutionOperation*, tensorflow::ImmediateExecutionTensorHandle**, int*) | |
| TFE_Execute | |
| TFE_Py_ExecuteCancelable(TFE_Context*, char const*, char const*, absl::lts_20211102::InlinedVector<TFE_TensorHandle*, 4ul, std::allocator<TFE_TensorHandle*> >*, _object*, TFE_CancellationManager*, absl::lts_20211102::InlinedVector<TFE_TensorHandle*, 2ul, std::allocator<TFE_TensorHandle*> >*, TF_Status*) | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call_Prepend | |
| _PyObject_MakeTpCall | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyObject_FastCallDictTstate | |
| _PyObject_Call | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| _PyEval_EvalFrameDefault | |
| PyEval_EvalCode | |
| _PyRun_SimpleFileObject | |
| _PyRun_AnyFileObject | |
| Py_RunMain | |
| Py_BytesMain | |
| __libc_start_main | |
| *** End stack trace *** | |
| == 2023-01-13 16:42:35,526 build_log.py:171 ERROR EasyBuild crashed with an error (at easybuild/software/EasyBuild/4.7.0/lib/python3.6/site-packages/easybuild/base/exceptions.py:126 in __init__): At least 3 cpu tests failed: | |
| //tensorflow/python/compiler/xla:jit_test_cpu, //tensorflow/python/data/kernel_tests:snapshot_test, //tensorflow/python/framework:constant_op_test (at easybuild/software/EasyBuild/4.7.0/lib/python3.6/site-packages/easybuild/framework/easyblock.py:2233 in report_test_failure) | |
| == 2023-01-13 16:42:35,527 build_log.py:267 INFO ... (took 3 hours 11 mins 3 secs) | |
| == 2023-01-13 16:42:35,527 build_log.py:267 INFO ... (took 3 hours 18 mins 1 secs) | |
| == 2023-01-13 16:42:35,527 filetools.py:2019 INFO Removing lock /users/jfg508/scratch/easybuild-dev/software/.locks/_users_jfg508_scratch_easybuild-dev_software_TensorFlow_2.9.1-foss-2022a-CUDA-11.7.0.lock... | |
| == 2023-01-13 16:42:35,529 filetools.py:384 INFO Path /users/jfg508/scratch/easybuild-dev/software/.locks/_users_jfg508_scratch_easybuild-dev_software_TensorFlow_2.9.1-foss-2022a-CUDA-11.7.0.lock successfully removed. | |
| == 2023-01-13 16:42:35,529 filetools.py:2023 INFO Lock removed: /users/jfg508/scratch/easybuild-dev/software/.locks/_users_jfg508_scratch_easybuild-dev_software_TensorFlow_2.9.1-foss-2022a-CUDA-11.7.0.lock | |
| == 2023-01-13 16:42:35,529 easyblock.py:4203 WARNING build failed (first 300 chars): At least 3 cpu tests failed: | |
| //tensorflow/python/compiler/xla:jit_test_cpu, //tensorflow/python/data/kernel_tests:snapshot_test, //tensorflow/python/framework:constant_op_test | |
| == 2023-01-13 16:42:35,529 easyblock.py:328 INFO Closing log for application name TensorFlow version 2.9.1 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment