摘要：

最近强化学习需要用到 tf-agents ，记录一下使用过程遇到的问题。

版本说明：

tensorflow                    2.14.0
gym                           0.23.0
tf-agents                     0.17.0
numpy                         1.24.0
ale-py                        0.8.1

报错命令：

from tf_agents.environments import suite_gym
env = suite_gym.load("Breakout-v4")
env.reset() # 报错 ValueError: setting an array element with a sequence

报错信息：

2023-10-24 21:53:43.710604: I tensorflow/core/platform/cpu_feature_guard.cc:182] This TensorFlow binary is optimized to use available CPU instructions in performance-critical operations.
To enable the following instructions: AVX2 AVX512F FMA, in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-10-24 21:53:44.618709: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Could not find TensorRT
A.L.E: Arcade Learning Environment (version 0.7.5+db37282)
[Powered by Stella]

Traceback (most recent call last):
  File "/home/ys/215/gaoshd/test.py", line 3, in <module>
    env.reset()
  File "/home/ys/.conda/envs/gaoshd_tf/lib/python3.9/site-packages/tf_agents/environments/py_environment.py", line 195, in reset
    self._current_time_step = self._reset()
  File "/home/ys/.conda/envs/gaoshd_tf/lib/python3.9/site-packages/tf_agents/environments/gym_wrapper.py", line 199, in _reset
    observation = self._to_obs_space_dtype(observation)
  File "/home/ys/.conda/envs/gaoshd_tf/lib/python3.9/site-packages/tf_agents/environments/gym_wrapper.py", line 246, in _to_obs_space_dtype
    matched_observations.append(np.asarray(obs, dtype=spec.dtype))
ValueError: setting an array element with a sequence. The requested array has an inhomogeneous shape after 1 dimensions. The detected shape was (2,) + inhomogeneous part.

解决方案：

该报错原因一直纠结于是 numpy 版本的问题，其实不然，原因是 ale-py 的版本问题，0.8.1 版本会引起  ValueError 问题，降低一个版本即可。

pip install ale-py==0.7.5