# Environment check throws Entire first structure: (., {}) Entire second structure:

**URL:** <https://discuss.ray.io/t/environment-check-throws-entire-first-structure-entire-second-structure/10738>\
**Category:** RLlib\
**Created:** [May 20, 2023, 9:33am UTC](https://discuss.ray.io/t/environment-check-throws-entire-first-structure-entire-second-structure/10738 "2023-05-20T09:33:47Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Athe-kunal](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/athe-kunal/32/1662_2.png) [@Athe-kunal](https://discuss.ray.io/u/Athe-kunal)\
**Post date:** [May 20, 2023, 9:33am UTC](https://discuss.ray.io/t/environment-check-throws-entire-first-structure-entire-second-structure/10738/1 "2023-05-20T09:33:47Z")

</div>

- Low: It annoys or frustrates me for a moment.  
I am trying to build this project [GitHub - Athe-kunal/Ray-tune-project](https://github.com/Athe-kunal/Ray-tune-project), where I am training an RL agent for stock market trading.  
In the `debug.py` file, [Ray-tune-project/debug.py at main · Athe-kunal/Ray-tune-project · GitHub](https://github.com/Athe-kunal/Ray-tune-project/blob/main/debug.py), I have given a simple script that uses RLlib to train on my StockMarketEnvironment. It works fine and it evaluates.

But when I do the same thing with Ray tune [Ray-tune-project/lstm.py at main · Athe-kunal/Ray-tune-project · GitHub](https://github.com/Athe-kunal/Ray-tune-project/blob/main/lstm.py#L127), it is failing with the following error

```auto
First structure: type=tuple str=(array([2.2788222e+02, 2.9840982e-01, 0.0000000e+00, 2.7170289e-01,
        1.4102402e+00, 1.2217970e+00, 1.8251243e+00, 1.0809784e+00,
        8.5718751e-01, 2.5914115e-01, 1.3132960e+00, 1.0967580e+00,
        8.3406353e-01, 2.3491893e+00, 1.0383877e+00, 1.1138891e+00,
        1.8886892e+00, 3.0702892e-01, 1.1083301e+00, 7.0789695e-01,
        4.7711730e-01, 1.1759629e+00, 1.6288950e+00, 5.5352813e-01,
        4.9202183e-01, 5.5303156e-01, 9.6180123e-01, 1.1277000e+00,
        1.0114505e+00, 8.0985188e-01, 4.9966720e-01, 6.7659909e-01,
        1.0117418e+00, 2.0312500e-01, 2.8125000e-01, 0.0000000e+00,
        2.8125000e-01, 7.8125000e-01, 7.9687500e-01, 9.3750000e-02,
        3.5937500e-01, 2.9687500e-01, 6.8750000e-01, 4.6875000e-02,
        5.4687500e-01, 2.8125000e-01, 6.8750000e-01, 1.2500000e-01,
        4.8437500e-01, 7.5000000e-01, 6.8750000e-01, 6.8750000e-01,
        5.3125000e-01, 2.1875000e-01, 8.4375000e-01, 6.4062500e-01,
        8.7500000e-01, 5.7812500e-01, 7.0312500e-01, 6.8750000e-01,
        9.2187500e-01, 9.8437500e-01, 2.8125000e-01, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.5433314e-01,
        1.5433314e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 9.0468752e-01,
        9.0468752e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.9882810e-01,
        6.9882810e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.0677344e+00,
        1.0677344e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 7.0210940e-01,
        7.0210940e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 4.2859375e-01,
        4.2859375e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.7187500e-01,
        1.7187500e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 9.6984375e-01,
        9.6984375e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 5.9585935e-01,
        5.9585935e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 4.4359374e-01,
        4.4359374e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.3819531e+00,
        1.3819531e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.4078122e-01,
        6.4078122e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.7272139e-01,
        6.7272139e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.3857105e+00,
        1.3857105e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 2.0148438e-01,
        2.0148438e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 7.1117187e-01,
        7.1117187e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 4.5476562e-01,
        4.5476562e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 3.1765625e-01,
        3.1765625e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 7.5320315e-01,
        7.5320315e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.0791407e+00,
        1.0791407e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 3.6893189e-01,
        3.6893189e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 2.9031250e-01,
        2.9031250e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 3.0562499e-01,
        3.0562499e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.2921876e-01,
        6.2921876e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.9789064e-01,
        6.9789064e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 5.8257812e-01,
        5.8257812e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 4.3166015e-01,
        4.3166015e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 3.8281250e-01,
        3.8281250e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 4.4242188e-01,
        4.4242188e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.1648440e-01,
        6.1648440e-01], dtype=float32), {})
 
 Second structure: type=ndarray str=[ -792.2486 -2144.3738 -129.80092 2915.458 -1308.43
   1269.5146 2074.0398 1901.5674 2501.5889 1164.0255
   -322.9323 1597.6952 -2573.6863 -2746.5698 2014.0723
  -1753.0637 -850.8143 2718.6846 2060.1797 258.4059
    545.0819 2851.2747 -1561.0944 349.4782 -1130.0878
   -403.55383 1070.6107 1372.8518 -2732.747 2289.9187
   2586.476 -1875.0045 -1937.3246 1576.0361 1310.3674
   -774.18475 -803.65875 2640.113 -840.6407 2802.322
     19.350145 2128.231 -979.46356 31.213558 -1505.1682
    208.86848 -780.9078 -1926.5431 2011.087 -629.80927
   1189.3198 2154.1665 2446.9292 -1517.6228 -167.33015
   -185.99898 1706.5778 824.91113 1430.8196 289.54074
    -83.10947 -2880.832 -1074.4584 2531.774 -290.6278
    329.9323 -1801.7854 -2637.3906 16.099054 -168.64241
   -125.876114 -1034.4384 1642.2655 351.62454 509.90973
    360.6945 -697.22485 -1484.3866 -1492.6287 -459.1929
  -2391.1506 -1728.1654 -764.98883 -1659.634 -2860.677
   -773.3472 2364.6418 916.4717 507.7752 1981.0371
   -257.88364 -637.90393 -901.27203 -1614.2391 -721.26996
   1833.0618 -1154.7478 969.5041 175.16115 223.94052
  -2363.3438 286.42517 1502.3491 -1839.2252 990.35913
   1965.7655 2190.7688 -74.32482 -1554.0822 -1204.554
    359.37643 1198.4722 -960.0168 1388.8052 1451.6226
  -2596.9067 -851.9495 -2175.7922 1587.5688 -322.92984
   1295.961 -2309.2183 2583.9165 2652.8699 2667.5493
   2502.1238 2813.4495 -842.62805 -242.12244 1219.3545
   2137.8694 -829.9557 1579.9534 2924.1921 -1282.7042
   2918.0173 -1075.8888 1481.3325 1201.7566 689.5607
  -1504.4034 1773.555 2896.91 2875.2483 -138.64302
  -2744.6174 -629.5382 1995.0635 -1050.1123 -73.988846
   -825.6564 -2097.2258 -809.13696 -2182.993 -2932.378
  -1921.0228 2604.5742 1624.0813 -2238.1482 -2156.0967
  -1990.5931 -2145.9197 2912.961 -726.6033 1492.1472
   1311.7047 -1292.6624 676.77026 -419.49207 -1351.4688
  -2679.772 920.6515 -1701.5579 -2236.0894 2608.5308
   -522.4806 -729.45245 -1825.2494 1263.7957 -882.4481
    338.66553 -2045.2618 -1575.5939 839.1834 -764.1029
   -687.5354 -2897.113 -2659.8728 -1605.8586 1714.4854
   1740.4427 2742.6748 367.4229 1100.9697 -2560.8313
   1445.6852 -1679.0385 -2801.7346 895.27704 1424.2039
  -2685.7834 634.448 1515.6146 -2839.2126 -2124.1313
   -331.8107 -2572.0156 2901.6733 1867.6338 -889.7915
   2168.2368 -2892.2544 2699.993 -1446.0782 -2005.3076
   -287.88196 -571.70496 -1412.371 1470.7764 1845.962
    321.17136 -32.960793 -486.28726 -2803.8381 -2929.3594
    934.08813 501.7609 391.37888 420.77762 -1653.7288
  -2707.625 -2399.6025 -514.26886 2206.8079 -1886.1824
   2690.6538 -398.5362 715.6277 1739.5406 -2561.3362
     74.25057 -945.2912 634.98395 663.10583 114.414795
    384.81158 -2277.277 1975.9971 1328.6526 819.96796
   1112.7996 55.950527 -1017.8577 -1035.914 1011.9785
    714.5928 1143.4572 -772.09064 1600.602 -283.70132
   -463.30814 -519.4983 -96.57187 662.2404 518.65125
    440.34192 -543.07623 1639.1079 1398.8292 2889.071
  -2044.9774 284.34378 1523.3135 -2078.8965 269.63492
   -990.6091 -844.06244 -1025.7518 769.9258 -2966.9124
  -1770.2169 -2029.7166 -11.077788 -1954.0278 146.9773
   1446.9406 2624.8562 2317.0337 51.89259 603.17914
  -1323.7025 1419.9907 -2526.2334 1230.5354 2883.4102
  -1753.4723 1794.5089 613.6288 2328.141 2853.8284
   2666.7334 -1498.047 1295.5944 967.1999 -2580.7476
  -2294.3696 1557.7621 2936.6091 -1691.9373 -1991.3083
   -895.6827 2094.7136 -1278.9231 -96.7892 -472.42078
   1660.7162 75.07274 -2942.1597 -1975.431 -2479.6326
   2608.9258 2222.379 1456.2847 -1376.8156 83.49044
   -281.9596 -1511.0402 -807.6998 -1640.7294 673.7634
  -1975.97 2338.9548 -2384.6924 ]
 
 More specifically: Substructure "type=tuple str=(array([2.2788222e+02, 2.9840982e-01, 0.0000000e+00, 2.7170289e-01,
        1.4102402e+00, 1.2217970e+00, 1.8251243e+00, 1.0809784e+00,
        8.5718751e-01, 2.5914115e-01, 1.3132960e+00, 1.0967580e+00,
        8.3406353e-01, 2.3491893e+00, 1.0383877e+00, 1.1138891e+00,
        1.8886892e+00, 3.0702892e-01, 1.1083301e+00, 7.0789695e-01,
        4.7711730e-01, 1.1759629e+00, 1.6288950e+00, 5.5352813e-01,
        4.9202183e-01, 5.5303156e-01, 9.6180123e-01, 1.1277000e+00,
        1.0114505e+00, 8.0985188e-01, 4.9966720e-01, 6.7659909e-01,
        1.0117418e+00, 2.0312500e-01, 2.8125000e-01, 0.0000000e+00,
        2.8125000e-01, 7.8125000e-01, 7.9687500e-01, 9.3750000e-02,
        3.5937500e-01, 2.9687500e-01, 6.8750000e-01, 4.6875000e-02,
        5.4687500e-01, 2.8125000e-01, 6.8750000e-01, 1.2500000e-01,
        4.8437500e-01, 7.5000000e-01, 6.8750000e-01, 6.8750000e-01,
        5.3125000e-01, 2.1875000e-01, 8.4375000e-01, 6.4062500e-01,
        8.7500000e-01, 5.7812500e-01, 7.0312500e-01, 6.8750000e-01,
        9.2187500e-01, 9.8437500e-01, 2.8125000e-01, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.5433314e-01,
        1.5433314e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 9.0468752e-01,
        9.0468752e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.9882810e-01,
        6.9882810e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.0677344e+00,
        1.0677344e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 7.0210940e-01,
        7.0210940e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 4.2859375e-01,
        4.2859375e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.7187500e-01,
        1.7187500e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 9.6984375e-01,
        9.6984375e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 5.9585935e-01,
        5.9585935e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 4.4359374e-01,
        4.4359374e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.3819531e+00,
        1.3819531e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.4078122e-01,
        6.4078122e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.7272139e-01,
        6.7272139e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.3857105e+00,
        1.3857105e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 2.0148438e-01,
        2.0148438e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 7.1117187e-01,
        7.1117187e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 4.5476562e-01,
        4.5476562e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 3.1765625e-01,
        3.1765625e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 7.5320315e-01,
        7.5320315e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 1.0791407e+00,
        1.0791407e+00, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 3.6893189e-01,
        3.6893189e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 2.9031250e-01,
        2.9031250e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 3.0562499e-01,
        3.0562499e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.2921876e-01,
        6.2921876e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.9789064e-01,
        6.9789064e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 5.8257812e-01,
        5.8257812e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 4.3166015e-01,
        4.3166015e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 3.8281250e-01,
        3.8281250e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 4.4242188e-01,
        4.4242188e-01, 0.0000000e+00, 0.0000000e+00, 0.0000000e+00,
        0.0000000e+00, 0.0000000e+00, 0.0000000e+00, 6.1648440e-01,
        6.1648440e-01], dtype=float32), {})" is a sequence, while substructure "type=ndarray str=[ -792.2486 -2144.3738 -129.80092 2915.458 -1308.43
   1269.5146 2074.0398 1901.5674 2501.5889 1164.0255
   -322.9323 1597.6952 -2573.6863 -2746.5698 2014.0723
  -1753.0637 -850.8143 2718.6846 2060.1797 258.4059
    545.0819 2851.2747 -1561.0944 349.4782 -1130.0878
   -403.55383 1070.6107 1372.8518 -2732.747 2289.9187
   2586.476 -1875.0045 -1937.3246 1576.0361 1310.3674
   -774.18475 -803.65875 2640.113 -840.6407 2802.322
     19.350145 2128.231 -979.46356 31.213558 -1505.1682
    208.86848 -780.9078 -1926.5431 2011.087 -629.80927
   1189.3198 2154.1665 2446.9292 -1517.6228 -167.33015
   -185.99898 1706.5778 824.91113 1430.8196 289.54074
    -83.10947 -2880.832 -1074.4584 2531.774 -290.6278
    329.9323 -1801.7854 -2637.3906 16.099054 -168.64241
   -125.876114 -1034.4384 1642.2655 351.62454 509.90973
    360.6945 -697.22485 -1484.3866 -1492.6287 -459.1929
  -2391.1506 -1728.1654 -764.98883 -1659.634 -2860.677
   -773.3472 2364.6418 916.4717 507.7752 1981.0371
   -257.88364 -637.90393 -901.27203 -1614.2391 -721.26996
   1833.0618 -1154.7478 969.5041 175.16115 223.94052
  -2363.3438 286.42517 1502.3491 -1839.2252 990.35913
   1965.7655 2190.7688 -74.32482 -1554.0822 -1204.554
    359.37643 1198.4722 -960.0168 1388.8052 1451.6226
  -2596.9067 -851.9495 -2175.7922 1587.5688 -322.92984
   1295.961 -2309.2183 2583.9165 2652.8699 2667.5493
   2502.1238 2813.4495 -842.62805 -242.12244 1219.3545
   2137.8694 -829.9557 1579.9534 2924.1921 -1282.7042
   2918.0173 -1075.8888 1481.3325 1201.7566 689.5607
  -1504.4034 1773.555 2896.91 2875.2483 -138.64302
  -2744.6174 -629.5382 1995.0635 -1050.1123 -73.988846
   -825.6564 -2097.2258 -809.13696 -2182.993 -2932.378
  -1921.0228 2604.5742 1624.0813 -2238.1482 -2156.0967
  -1990.5931 -2145.9197 2912.961 -726.6033 1492.1472
   1311.7047 -1292.6624 676.77026 -419.49207 -1351.4688
  -2679.772 920.6515 -1701.5579 -2236.0894 2608.5308
   -522.4806 -729.45245 -1825.2494 1263.7957 -882.4481
    338.66553 -2045.2618 -1575.5939 839.1834 -764.1029
   -687.5354 -2897.113 -2659.8728 -1605.8586 1714.4854
   1740.4427 2742.6748 367.4229 1100.9697 -2560.8313
   1445.6852 -1679.0385 -2801.7346 895.27704 1424.2039
  -2685.7834 634.448 1515.6146 -2839.2126 -2124.1313
   -331.8107 -2572.0156 2901.6733 1867.6338 -889.7915
   2168.2368 -2892.2544 2699.993 -1446.0782 -2005.3076
   -287.88196 -571.70496 -1412.371 1470.7764 1845.962
    321.17136 -32.960793 -486.28726 -2803.8381 -2929.3594
    934.08813 501.7609 391.37888 420.77762 -1653.7288
  -2707.625 -2399.6025 -514.26886 2206.8079 -1886.1824
   2690.6538 -398.5362 715.6277 1739.5406 -2561.3362
     74.25057 -945.2912 634.98395 663.10583 114.414795
    384.81158 -2277.277 1975.9971 1328.6526 819.96796
   1112.7996 55.950527 -1017.8577 -1035.914 1011.9785
    714.5928 1143.4572 -772.09064 1600.602 -283.70132
   -463.30814 -519.4983 -96.57187 662.2404 518.65125
    440.34192 -543.07623 1639.1079 1398.8292 2889.071
  -2044.9774 284.34378 1523.3135 -2078.8965 269.63492
   -990.6091 -844.06244 -1025.7518 769.9258 -2966.9124
  -1770.2169 -2029.7166 -11.077788 -1954.0278 146.9773
   1446.9406 2624.8562 2317.0337 51.89259 603.17914
  -1323.7025 1419.9907 -2526.2334 1230.5354 2883.4102
  -1753.4723 1794.5089 613.6288 2328.141 2853.8284
   2666.7334 -1498.047 1295.5944 967.1999 -2580.7476
  -2294.3696 1557.7621 2936.6091 -1691.9373 -1991.3083
   -895.6827 2094.7136 -1278.9231 -96.7892 -472.42078
   1660.7162 75.07274 -2942.1597 -1975.431 -2479.6326
   2608.9258 2222.379 1456.2847 -1376.8156 83.49044
   -281.9596 -1511.0402 -807.6998 -1640.7294 673.7634
  -1975.97 2338.9548 -2384.6924 ]" is not
 Entire first structure:
 (., {})
 Entire second structure:
 .

```

My guess:  
The first structure probably comes from the reset function, and the second structure from the step function. As per gymnasium API, we need to have an info\_dict in the reset function, so that’s why we have an extra {}, but If I remove it, then gymnasium complains. This environment [Ray-tune-project/env\_stocktrading\_np.py at main · Athe-kunal/Ray-tune-project · GitHub](https://github.com/Athe-kunal/Ray-tune-project/blob/main/env_stocktrading_np.py) is working fine with RLlib for both LSTM and Attention network, but it is failing with Ray tune. This was working a month back, but now it is showing this issue. Am I missing something here? Please let me know, and if something is not clear, let me know, I will explain it again.

---

<div class="post-metadata">

**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\
**Post date:** [May 20, 2023, 5:25pm UTC](https://discuss.ray.io/t/environment-check-throws-entire-first-structure-entire-second-structure/10738/2 "2023-05-20T17:25:17Z")

</div>

Please post a minimal reproduction script! This is hard to work on otherwise

---

<div class="post-metadata">

**Author:** ![Athe-kunal](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/athe-kunal/32/1662_2.png) [@Athe-kunal](https://discuss.ray.io/u/Athe-kunal)\
**Post date:** [May 21, 2023, 8:04am UTC](https://discuss.ray.io/t/environment-check-throws-entire-first-structure-entire-second-structure/10738/3 "2023-05-21T08:04:50Z")

</div>

Hi @arturn, sorry for not giving the reproduction script. Here is the reproduction script.  
PLEASE ENSURE THAT YOU DOWNLOAD THESE TWO FILES FIRST BEFORE RUNNING THE SCRIPT, AS THE DATA IS REQUIRED FOR THE ENVIRONMENT.

> <https://github.com/Athe-kunal/Ray-tune-project/blob/main/train.npy>

> <https://github.com/Athe-kunal/Ray-tune-project/blob/main/test.npy>

```auto
from __future__ import annotations

import gymnasium as gym
import numpy as np
from numpy import random as rd

class StockTradingEnv(gym.Env):
    def __init__ (
        self,
        config,
        initial_account=1e6,
        gamma=0.99,
        turbulence_thresh=99,
        min_stock_rate=0.1,
        max_stock=1e2,
        initial_capital=1e6,
        buy_cost_pct=1e-3,
        sell_cost_pct=1e-3,
        reward_scaling=2**-11,
        initial_stocks=None,
    ):
        price_ary = config["price_array"]
        tech_ary = config["tech_array"]
        turbulence_ary = config["turbulence_array"]
        if_train = config["if_train"]
        self.price_ary = price_ary.astype(np.float32)
        self.tech_ary = tech_ary.astype(np.float32)
        self.turbulence_ary = turbulence_ary

        self.tech_ary = self.tech_ary * 2**-7
        self.turbulence_bool = (turbulence_ary > turbulence_thresh).astype(np.float32)
        self.turbulence_ary = (
            self.sigmoid_sign(turbulence_ary, turbulence_thresh) * 2**-5
        ).astype(np.float32)

        stock_dim = self.price_ary.shape[1]
        self.gamma = gamma
        self.max_stock = max_stock
        self.min_stock_rate = min_stock_rate
        self.buy_cost_pct = buy_cost_pct
        self.sell_cost_pct = sell_cost_pct
        self.reward_scaling = reward_scaling
        self.initial_capital = initial_capital
        self.initial_stocks = (
            np.zeros(stock_dim, dtype=np.float32)
            if initial_stocks is None
            else initial_stocks
        )

        # reset()
        self.day = None
        self.amount = None
        self.stocks = None
        self.total_asset = None
        self.gamma_reward = None
        self.initial_total_asset = None

        # environment information
        self.env_name = "StockEnv"
        # self.state_dim = 1 + 2 + 2 * stock_dim + self.tech_ary.shape[1]
        # # amount + (turbulence, turbulence_bool) + (price, stock) * stock_dim + tech_dim
        self.state_dim = 1 + 2 + 3 * stock_dim + self.tech_ary.shape[1]
        # amount + (turbulence, turbulence_bool) + (price, stock) * stock_dim + tech_dim
        self.stocks_cd = None
        self.action_dim = stock_dim
        self.max_step = self.price_ary.shape[0] - 1
        self.if_train = if_train
        self.if_discrete = False
        self.target_return = 10.0
        self.episode_return = 0.0

        self.observation_space = gym.spaces.Box(
            low=-3000, high=3000, shape=(self.state_dim,), dtype=np.float32
        )
        self.action_space = gym.spaces.Box(
            low=-1, high=1, shape=(self.action_dim,), dtype=np.float32
        )

    def reset(self, seed=None, options=None):
        self.day = 0
        price = self.price_ary[self.day]

        if self.if_train:
            self.stocks = (
                self.initial_stocks + rd.randint(0, 64, size=self.initial_stocks.shape)
            ).astype(np.float32)
            self.stocks_cool_down = np.zeros_like(self.stocks)
            self.amount = (
                self.initial_capital * rd.uniform(0.95, 1.05)
                - (self.stocks * price).sum()
            )
        else:
            self.stocks = self.initial_stocks.astype(np.float32)
            self.stocks_cool_down = np.zeros_like(self.stocks)
            self.amount = self.initial_capital

        self.total_asset = self.amount + (self.stocks * price).sum()
        self.initial_total_asset = self.total_asset
        self.gamma_reward = 0.0
        print(self.get_state(price).shape)
        return self.get_state(price), {} # state

    def step(self, actions):
        actions = (actions * self.max_stock).astype(int)

        self.day += 1
        price = self.price_ary[self.day]
        self.stocks_cool_down += 1

        if self.turbulence_bool[self.day] == 0:
            min_action = int(self.max_stock * self.min_stock_rate) # stock_cd
            for index in np.where(actions < -min_action)[0]: # sell_index:
                if price[index] > 0: # Sell only if current asset is > 0
                    sell_num_shares = min(self.stocks[index], -actions[index])
                    self.stocks[index] -= sell_num_shares
                    self.amount += (
                        price[index] * sell_num_shares * (1 - self.sell_cost_pct)
                    )
                    self.stocks_cool_down[index] = 0
            for index in np.where(actions > min_action)[0]: # buy_index:
                if (
                    price[index] > 0
                ): # Buy only if the price is > 0 (no missing data in this particular date)
                    buy_num_shares = min(self.amount // price[index], actions[index])
                    self.stocks[index] += buy_num_shares
                    self.amount -= (
                        price[index] * buy_num_shares * (1 + self.buy_cost_pct)
                    )
                    self.stocks_cool_down[index] = 0

        else: # sell all when turbulence
            self.amount += (self.stocks * price).sum() * (1 - self.sell_cost_pct)
            self.stocks[:] = 0
            self.stocks_cool_down[:] = 0

        state = self.get_state(price)
        print(state.shape)
        total_asset = self.amount + (self.stocks * price).sum()
        reward = (total_asset - self.total_asset) * self.reward_scaling
        self.total_asset = total_asset

        self.gamma_reward = self.gamma_reward * self.gamma + reward
        done = self.day == self.max_step
        if done:
            reward = self.gamma_reward
            self.episode_return = total_asset / self.initial_total_asset

        return state, reward, done, done, dict()

    def get_state(self, price):
        amount = np.array(self.amount * (2**-12), dtype=np.float32)
        scale = np.array(2**-6, dtype=np.float32)
        return np.hstack(
            (
                amount,
                self.turbulence_ary[self.day],
                self.turbulence_bool[self.day],
                price * scale,
                self.stocks * scale,
                self.stocks_cool_down,
                self.tech_ary[self.day],
            )
        ) # state.astype(np.float32)

    @staticmethod
    def sigmoid_sign(ary, thresh):
        def sigmoid(x):
            return 1 / (1 + np.exp(-x * np.e)) - 0.5

        return sigmoid(ary / thresh) * thresh

import ray

assert (
    ray. __version__ > "2.0.0"
), "Please install ray 2.2.0 by doing 'pip install ray[rllib] ray[tune] lz4' , lz4 is for population based tuning"
from pprint import pprint

import psutil
from ray import tune
from ray.air import FailureConfig, RunConfig, ScalingConfig
from ray.air.config import CheckpointConfig
from ray.rllib.algorithms import Algorithm
from ray.tune import register_env
from ray.tune.callback import Callback
from ray.tune.search import ConcurrencyLimiter
from ray.tune.tune_config import TuneConfig

psutil_memory_in_bytes = psutil.virtual_memory().total
ray._private.utils.get_system_memory = lambda: psutil_memory_in_bytes
from typing import Any, Dict, List, Optional, Union

class DRLlibv2:
    def __init__ (
        self,
        trainable: Union[str, Any],
        params: dict,
        train_env=None,
        train_env_name: str = "",
        run_name: str = "tune_run",
        framework: str = "torch",
        local_dir: str = "tune_results",
        num_workers: int = 1,
        search_alg=None,
        concurrent_trials: int = 0,
        num_samples: int = 0,
        scheduler=None,
        log_level: str = "WARN",
        num_gpus: Union[float, int] = 0,
        num_cpus: Union[float, int] = 2,
        dataframe_save: str = "tune.csv",
        metric: str = "episode_reward_mean",
        mode: Union[str, List[str]] = "max",
        max_failures: int = 0,
        training_iterations: int = 100,
        checkpoint_num_to_keep: Union[None, int] = None,
        checkpoint_freq: int = 0,
        reuse_actors: bool = False,
        callbacks: Optional[List["Callback"]] = None,
    ):

        if train_env is not None:
            register_env(train_env_name, lambda config: train_env)

        self.params = params
        self.params["framework"] = framework
        self.params["log_level"] = log_level
        self.params["num_gpus"] = num_gpus
        self.params["num_workers"] = num_workers
        self.params["env"] = train_env_name

        self.run_name = run_name
        self.local_dir = local_dir
        self.search_alg = search_alg
        if concurrent_trials != 0:
            self.search_alg = ConcurrencyLimiter(
                self.search_alg, max_concurrent=concurrent_trials
            )
        self.scheduler = scheduler
        self.num_samples = num_samples
        self.trainable = trainable
        # self.trainable = tune.with_resources(self.trainable(),{"cpu":num_cpus,"gpu":num_gpus})
        if isinstance(self.trainable, str):
            self.trainable.upper()
        self.num_cpus = num_cpus
        self.num_gpus = num_gpus
        self.dataframe_save = dataframe_save
        self.metric = metric
        self.mode = mode
        self.max_failures = max_failures
        self.training_iterations = training_iterations
        self.checkpoint_freq = checkpoint_freq
        self.checkpoint_num_to_keep = checkpoint_num_to_keep
        self.reuse_actors = reuse_actors
        self.callbacks = callbacks

    def train_tune_model(self):
        """
        Tuning and training the model
        Returns the results object
        """
        ray.init(
            num_cpus=self.num_cpus, num_gpus=self.num_gpus, ignore_reinit_error=True
        )

        tuner = tune.Tuner(
            self.trainable,
            param_space=self.params,
            tune_config=TuneConfig(
                search_alg=self.search_alg,
                num_samples=self.num_samples,
                metric=self.metric,
                mode=self.mode,
                reuse_actors=self.reuse_actors,
            ),
            run_config=RunConfig(
                name=self.run_name,
                local_dir=self.local_dir,
                callbacks=self.callbacks,
                failure_config=FailureConfig(
                    max_failures=self.max_failures, fail_fast=False
                ),
                stop={"training_iteration": self.training_iterations},
                checkpoint_config=CheckpointConfig(
                    num_to_keep=self.checkpoint_num_to_keep,
                    checkpoint_score_attribute=self.metric,
                    checkpoint_score_order=self.mode,
                    checkpoint_frequency=self.checkpoint_freq,
                    checkpoint_at_end=True,
                ),
                verbose=3,
            ),
        )

        self.results = tuner.fit()
        # if self.search_alg is not None: self.search_alg.save_to_dir(self.local_dir)
        # ray.shutdown()
        return self.results

import ray

assert (
    ray. __version__ >= "2.2.0"
), "Please install ray 2.2.0 by doing 'pip install ray[rllib] ray[tune] lz4' , lz4 is for population based tuning"

from pprint import pprint

import numpy as np
# Importing the libraries
import pandas as pd
import psutil
import ray
import ray.rllib.algorithms.ppo as ppo
from ray import tune
from ray.tune.registry import register_env
from ray.tune.schedulers import ASHAScheduler
from ray.tune.search import ConcurrencyLimiter
from ray.tune.search.optuna import OptunaSearch

psutil_memory_in_bytes = psutil.virtual_memory().total
ray._private.utils.get_system_memory = lambda: psutil_memory_in_bytes
from typing import Any, Dict, Optional

model_name = "PPO"
env = StockTradingEnv
data_source = "yahoofinance"
time_interval = "1D"

import warnings

warnings.filterwarnings("ignore", category=FutureWarning)

import numpy as np

with open("train.npy", "rb") as f:
    train_env_config = np.load(f, allow_pickle=True)

with open("test.npy", "rb") as f:
    test_env_config = np.load(f, allow_pickle=True)
train_env_config = train_env_config.item()
test_env_config = test_env_config.item()
from gymnasium.wrappers import EnvCompatibility
from ray.tune import register_env

env_name = "StockTrading_train_env"

def reg_env(config):
    return env(config)

register_env(env_name, lambda config: env(train_env_config))

train_env_instance = EnvCompatibility(env(train_env_config))

def sample_ppo_params():
    return {
        "entropy_coeff": tune.loguniform(0.00000001, 0.1),
        "lr": tune.loguniform(5e-5, 0.001),
        "sgd_minibatch_size": tune.choice([32, 64, 128, 256, 512]),
        "lambda": tune.choice([0.1, 0.3, 0.5, 0.7, 0.9, 1.0]),
        # "entropy_coeff": 0.0000001,
        # "lr": 5e-5,
        # "sgd_minibatch_size": 64,
        # "lambda":0.9,
        "framework": "torch",
        "model": {
            "use_attention": True,
            "attention_num_transformer_units": 1,
            "attention_dim": 64,
            "attention_num_heads": 1,
            "attention_head_dim": 32,
            "attention_memory_inference": 50,
            "attention_memory_training": 50,
            "attention_position_wise_mlp_dim": 32,
            "attention_init_gru_gate_bias": 2.0,
            "attention_use_n_prev_actions": 0,
            "attention_use_n_prev_rewards": 0,
        },
    }

metric = "episode_reward_mean"
mode = "max"

search_alg = OptunaSearch(metric=metric, mode=mode)

scheduler_ = ASHAScheduler(
    metric=metric,
    mode=mode,
    max_t=5,
    grace_period=1,
    reduction_factor=2,
)

drl_agent = DRLlibv2(
    trainable=model_name,
    train_env=train_env_instance,
    train_env_name="StockTrading_train",
    framework="torch",
    num_workers=1,
    log_level="DEBUG",
    run_name="FINRL_TEST_TRANS",
    local_dir="FINRL_TEST_TRANS",
    params=sample_ppo_params(),
    num_samples=2,
    num_gpus=1,
    training_iterations=10,
    checkpoint_freq=5,
    scheduler=scheduler_,
    search_alg=search_alg,
)

trans_res = drl_agent.train_tune_model()

```

Ray: 2.4.0  
System: Windows WSL2

Please let me know if I missing something, thanks

---

<div class="post-metadata">

**Author:** ![arturn](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/arturn/32/2096_2.png) [@arturn](https://discuss.ray.io/u/arturn)\
**Post date:** [May 23, 2023, 6:08pm UTC](https://discuss.ray.io/t/environment-check-throws-entire-first-structure-entire-second-structure/10738/4 "2023-05-23T18:08:40Z")

</div>

Thanks for the script.  
The issue is that your wrap is compatible with the gymnasium API (for example, it has a reset method that returns observation and info dict), but still use `from gymnasium.wrappers import EnvCompatibility`.  
This wraps your environment and the wrapped environment returns `((obs, {}), {})`, while it should return `(obs, {})`

---

<div class="post-metadata">

**Author:** ![Athe-kunal](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/athe-kunal/32/1662_2.png) [@Athe-kunal](https://discuss.ray.io/u/Athe-kunal)\
**Post date:** [May 24, 2023, 6:25am UTC](https://discuss.ray.io/t/environment-check-throws-entire-first-structure-entire-second-structure/10738/5 "2023-05-24T06:25:58Z")

</div>

Thanks a lot, @arturn  
This was a savior, it fixed my issue.