HF中国镜像站's logo

francescosabbarese
/
rl_course_vizdoom_health_gathering_supreme

Reinforcement Learning
sample-factory
TensorBoard
deep-reinforcement-learning
Eval Results
Model card Files Metrics Community
francescosabbarese commited on
Commit
4adca3c
·
verified ·
1 Parent(s): 5df5f80

Upload folder using huggingface_hub

Browse files
Files changed (9) hide show
  1. .gitattributes +1 -0
  2. .summary/0/events.out.tfevents.1740595416.97a490883127 +3 -0
  3. README.md +56 -0
  4. checkpoint_p0/best_000000842_3448832_reward_6.318.pth +3 -0
  5. checkpoint_p0/checkpoint_000000783_3207168.pth +3 -0
  6. checkpoint_p0/checkpoint_000000978_4005888.pth +3 -0
  7. config.json +147 -0
  8. replay.mp4 +3 -0
  9. sf_log.txt +730 -0
.gitattributes CHANGED
@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ replay.mp4 filter=lfs diff=lfs merge=lfs -text
.summary/0/events.out.tfevents.1740595416.97a490883127 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:77a6e999c44e18e49908bbd7ff54ce331cdb20961d3567344da5891b64cf977b
3
+ size 391953
README.md ADDED
@@ -0,0 +1,56 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: sample-factory
3
+ tags:
4
+ - deep-reinforcement-learning
5
+ - reinforcement-learning
6
+ - sample-factory
7
+ model-index:
8
+ - name: APPO
9
+ results:
10
+ - task:
11
+ type: reinforcement-learning
12
+ name: reinforcement-learning
13
+ dataset:
14
+ name: doom_health_gathering_supreme
15
+ type: doom_health_gathering_supreme
16
+ metrics:
17
+ - type: mean_reward
18
+ value: 4.45 +/- 1.06
19
+ name: mean_reward
20
+ verified: false
21
+ ---
22
+
23
+ A(n) **APPO** model trained on the **doom_health_gathering_supreme** environment.
24
+
25
+ This model was trained using Sample-Factory 2.0: https://github.com/alex-petrenko/sample-factory.
26
+ Documentation for how to use Sample-Factory can be found at https://www.samplefactory.dev/
27
+
28
+
29
+ ## Downloading the model
30
+
31
+ After installing Sample-Factory, download the model with:
32
+ ```
33
+ python -m sample_factory.huggingface.load_from_hub -r francescosabbarese/rl_course_vizdoom_health_gathering_supreme
34
+ ```
35
+
36
+
37
+ ## Using the model
38
+
39
+ To run the model after download, use the `enjoy` script corresponding to this environment:
40
+ ```
41
+ python -m <path.to.enjoy.module> --algo=APPO --env=doom_health_gathering_supreme --train_dir=./train_dir --experiment=rl_course_vizdoom_health_gathering_supreme
42
+ ```
43
+
44
+
45
+ You can also upload models to the HF中国镜像站 Hub using the same script with the `--push_to_hub` flag.
46
+ See https://www.samplefactory.dev/10-huggingface/huggingface/ for more details
47
+
48
+ ## Training with this model
49
+
50
+ To continue training with this model, use the `train` script corresponding to this environment:
51
+ ```
52
+ python -m <path.to.train.module> --algo=APPO --env=doom_health_gathering_supreme --train_dir=./train_dir --experiment=rl_course_vizdoom_health_gathering_supreme --restart_behavior=resume --train_for_env_steps=10000000000
53
+ ```
54
+
55
+ Note, you may have to adjust `--train_for_env_steps` to a suitably high number as the experiment will resume at the number of steps it concluded at.
56
+
checkpoint_p0/best_000000842_3448832_reward_6.318.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:13b517c2d8d63caf58336370ba62486008cff31d2da4940094e87d76d682de93
3
+ size 21523663
checkpoint_p0/checkpoint_000000783_3207168.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1d95f6e5135e25b1d21be791d66aaf0c5af7d07d9d612a09438ce29bc63c0b08
3
+ size 21524161
checkpoint_p0/checkpoint_000000978_4005888.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8f500e5df5fc8db1c8f2a61d874ed997c8a7e6fd79a167f9a13c4c1e5a236c7b
3
+ size 21524161
config.json ADDED
@@ -0,0 +1,147 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "help": false,
3
+ "algo": "APPO",
4
+ "env": "doom_health_gathering_supreme",
5
+ "experiment": "default_experiment",
6
+ "train_dir": "/kaggle/working/train_dir",
7
+ "restart_behavior": "resume",
8
+ "device": "gpu",
9
+ "seed": null,
10
+ "num_policies": 1,
11
+ "async_rl": true,
12
+ "serial_mode": false,
13
+ "batched_sampling": false,
14
+ "num_batches_to_accumulate": 2,
15
+ "worker_num_splits": 2,
16
+ "policy_workers_per_policy": 1,
17
+ "max_policy_lag": 1000,
18
+ "num_workers": 8,
19
+ "num_envs_per_worker": 8,
20
+ "batch_size": 1024,
21
+ "num_batches_per_epoch": 1,
22
+ "num_epochs": 1,
23
+ "rollout": 32,
24
+ "recurrence": 32,
25
+ "shuffle_minibatches": false,
26
+ "gamma": 0.98,
27
+ "reward_scale": 1.0,
28
+ "reward_clip": 1000.0,
29
+ "value_bootstrap": false,
30
+ "normalize_returns": true,
31
+ "exploration_loss_coeff": 0.001,
32
+ "value_loss_coeff": 0.5,
33
+ "kl_loss_coeff": 0.0,
34
+ "exploration_loss": "symmetric_kl",
35
+ "gae_lambda": 0.95,
36
+ "ppo_clip_ratio": 0.1,
37
+ "ppo_clip_value": 0.2,
38
+ "with_vtrace": false,
39
+ "vtrace_rho": 1.0,
40
+ "vtrace_c": 1.0,
41
+ "optimizer": "adam",
42
+ "adam_eps": 1e-06,
43
+ "adam_beta1": 0.9,
44
+ "adam_beta2": 0.999,
45
+ "max_grad_norm": 4.0,
46
+ "learning_rate": 0.0001,
47
+ "lr_schedule": "constant",
48
+ "lr_schedule_kl_threshold": 0.008,
49
+ "lr_adaptive_min": 1e-06,
50
+ "lr_adaptive_max": 0.01,
51
+ "obs_subtract_mean": 0.0,
52
+ "obs_scale": 255.0,
53
+ "normalize_input": true,
54
+ "normalize_input_keys": null,
55
+ "decorrelate_experience_max_seconds": 0,
56
+ "decorrelate_envs_on_one_worker": true,
57
+ "actor_worker_gpus": [],
58
+ "set_workers_cpu_affinity": true,
59
+ "force_envs_single_thread": false,
60
+ "default_niceness": 0,
61
+ "log_to_file": true,
62
+ "experiment_summaries_interval": 10,
63
+ "flush_summaries_interval": 30,
64
+ "stats_avg": 100,
65
+ "summaries_use_frameskip": true,
66
+ "heartbeat_interval": 20,
67
+ "heartbeat_reporting_interval": 600,
68
+ "train_for_env_steps": 4000000,
69
+ "train_for_seconds": 10000000000,
70
+ "save_every_sec": 120,
71
+ "keep_checkpoints": 2,
72
+ "load_checkpoint_kind": "latest",
73
+ "save_milestones_sec": -1,
74
+ "save_best_every_sec": 5,
75
+ "save_best_metric": "reward",
76
+ "save_best_after": 100000,
77
+ "benchmark": false,
78
+ "encoder_mlp_layers": [
79
+ 512,
80
+ 512
81
+ ],
82
+ "encoder_conv_architecture": "convnet_simple",
83
+ "encoder_conv_mlp_layers": [
84
+ 512
85
+ ],
86
+ "use_rnn": true,
87
+ "rnn_size": 128,
88
+ "rnn_type": "lstm",
89
+ "rnn_num_layers": 2,
90
+ "decoder_mlp_layers": [],
91
+ "nonlinearity": "elu",
92
+ "policy_initialization": "xavier_uniform",
93
+ "policy_init_gain": 1.0,
94
+ "actor_critic_share_weights": true,
95
+ "adaptive_stddev": true,
96
+ "continuous_tanh_scale": 0.0,
97
+ "initial_stddev": 1.0,
98
+ "use_env_info_cache": false,
99
+ "env_gpu_actions": false,
100
+ "env_gpu_observations": true,
101
+ "env_frameskip": 4,
102
+ "env_framestack": 1,
103
+ "pixel_format": "CHW",
104
+ "use_record_episode_statistics": false,
105
+ "with_wandb": false,
106
+ "wandb_user": null,
107
+ "wandb_project": "sample_factory",
108
+ "wandb_group": null,
109
+ "wandb_job_type": "SF",
110
+ "wandb_tags": [],
111
+ "with_pbt": false,
112
+ "pbt_mix_policies_in_one_env": true,
113
+ "pbt_period_env_steps": 5000000,
114
+ "pbt_start_mutation": 20000000,
115
+ "pbt_replace_fraction": 0.3,
116
+ "pbt_mutation_rate": 0.15,
117
+ "pbt_replace_reward_gap": 0.1,
118
+ "pbt_replace_reward_gap_absolute": 1e-06,
119
+ "pbt_optimize_gamma": false,
120
+ "pbt_target_objective": "true_objective",
121
+ "pbt_perturb_min": 1.1,
122
+ "pbt_perturb_max": 1.5,
123
+ "num_agents": -1,
124
+ "num_humans": 0,
125
+ "num_bots": -1,
126
+ "start_bot_difficulty": null,
127
+ "timelimit": null,
128
+ "res_w": 128,
129
+ "res_h": 72,
130
+ "wide_aspect_ratio": false,
131
+ "eval_env_frameskip": 1,
132
+ "fps": 35,
133
+ "command_line": "--env=doom_health_gathering_supreme --num_workers=8 --num_envs_per_worker=8 --train_for_env_steps=4000000 --gamma=0.98 --rnn_type=lstm --policy_initialization=xavier_uniform --rnn_size=128 --rnn_num_layers=2",
134
+ "cli_args": {
135
+ "env": "doom_health_gathering_supreme",
136
+ "num_workers": 8,
137
+ "num_envs_per_worker": 8,
138
+ "gamma": 0.98,
139
+ "train_for_env_steps": 4000000,
140
+ "rnn_size": 128,
141
+ "rnn_type": "lstm",
142
+ "rnn_num_layers": 2,
143
+ "policy_initialization": "xavier_uniform"
144
+ },
145
+ "git_hash": "unknown",
146
+ "git_repo_name": "not a git repository"
147
+ }
replay.mp4 ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d28f72e1c38891a14420c3e43904ef6150e8513e0ddfb21e060f667ec4c04379
3
+ size 6339474
sf_log.txt ADDED
@@ -0,0 +1,730 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ [2025-02-26 18:43:41,668][00031] Saving configuration to /kaggle/working/train_dir/default_experiment/config.json...
2
+ [2025-02-26 18:43:41,670][00031] Rollout worker 0 uses device cpu
3
+ [2025-02-26 18:43:41,671][00031] Rollout worker 1 uses device cpu
4
+ [2025-02-26 18:43:41,672][00031] Rollout worker 2 uses device cpu
5
+ [2025-02-26 18:43:41,673][00031] Rollout worker 3 uses device cpu
6
+ [2025-02-26 18:43:41,674][00031] Rollout worker 4 uses device cpu
7
+ [2025-02-26 18:43:41,675][00031] Rollout worker 5 uses device cpu
8
+ [2025-02-26 18:43:41,676][00031] Rollout worker 6 uses device cpu
9
+ [2025-02-26 18:43:41,677][00031] Rollout worker 7 uses device cpu
10
+ [2025-02-26 18:43:41,871][00031] Using GPUs [0] for process 0 (actually maps to GPUs [0])
11
+ [2025-02-26 18:43:41,872][00031] InferenceWorker_p0-w0: min num requests: 2
12
+ [2025-02-26 18:43:41,915][00031] Starting all processes...
13
+ [2025-02-26 18:43:41,915][00031] Starting process learner_proc0
14
+ [2025-02-26 18:43:42,010][00031] Starting all processes...
15
+ [2025-02-26 18:43:42,018][00031] Starting process inference_proc0-0
16
+ [2025-02-26 18:43:42,019][00031] Starting process rollout_proc0
17
+ [2025-02-26 18:43:42,021][00031] Starting process rollout_proc1
18
+ [2025-02-26 18:43:42,021][00031] Starting process rollout_proc2
19
+ [2025-02-26 18:43:42,025][00031] Starting process rollout_proc3
20
+ [2025-02-26 18:43:42,025][00031] Starting process rollout_proc4
21
+ [2025-02-26 18:43:42,026][00031] Starting process rollout_proc5
22
+ [2025-02-26 18:43:42,026][00031] Starting process rollout_proc6
23
+ [2025-02-26 18:43:42,028][00031] Starting process rollout_proc7
24
+ [2025-02-26 18:43:49,327][01171] Worker 6 uses CPU cores [2]
25
+ [2025-02-26 18:43:50,115][01170] Worker 5 uses CPU cores [1]
26
+ [2025-02-26 18:43:50,116][01164] Using GPUs [0] for process 0 (actually maps to GPUs [0])
27
+ [2025-02-26 18:43:50,117][01164] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for inference process 0
28
+ [2025-02-26 18:43:50,175][01164] Num visible devices: 1
29
+ [2025-02-26 18:43:50,334][01172] Worker 7 uses CPU cores [3]
30
+ [2025-02-26 18:43:50,392][01151] Using GPUs [0] for process 0 (actually maps to GPUs [0])
31
+ [2025-02-26 18:43:50,393][01151] Set environment var CUDA_VISIBLE_DEVICES to '0' (GPU indices [0]) for learning process 0
32
+ [2025-02-26 18:43:50,422][01151] Num visible devices: 1
33
+ [2025-02-26 18:43:50,438][01151] Starting seed is not provided
34
+ [2025-02-26 18:43:50,439][01151] Using GPUs [0] for process 0 (actually maps to GPUs [0])
35
+ [2025-02-26 18:43:50,439][01151] Initializing actor-critic model on device cuda:0
36
+ [2025-02-26 18:43:50,440][01151] RunningMeanStd input shape: (3, 72, 128)
37
+ [2025-02-26 18:43:50,441][01168] Worker 3 uses CPU cores [3]
38
+ [2025-02-26 18:43:50,451][01151] RunningMeanStd input shape: (1,)
39
+ [2025-02-26 18:43:50,472][01165] Worker 0 uses CPU cores [0]
40
+ [2025-02-26 18:43:50,491][01151] ConvEncoder: input_channels=3
41
+ [2025-02-26 18:43:50,505][01166] Worker 1 uses CPU cores [1]
42
+ [2025-02-26 18:43:50,547][01167] Worker 2 uses CPU cores [2]
43
+ [2025-02-26 18:43:50,570][01169] Worker 4 uses CPU cores [0]
44
+ [2025-02-26 18:43:50,735][01151] Conv encoder output size: 512
45
+ [2025-02-26 18:43:50,735][01151] Policy head output size: 512
46
+ [2025-02-26 18:43:50,740][01151] Created Actor Critic model with architecture:
47
+ [2025-02-26 18:43:50,740][01151] ActorCriticSharedWeights(
48
+ (obs_normalizer): ObservationNormalizer(
49
+ (running_mean_std): RunningMeanStdDictInPlace(
50
+ (running_mean_std): ModuleDict(
51
+ (obs): RunningMeanStdInPlace()
52
+ )
53
+ )
54
+ )
55
+ (returns_normalizer): RecursiveScriptModule(original_name=RunningMeanStdInPlace)
56
+ (encoder): VizdoomEncoder(
57
+ (basic_encoder): ConvEncoder(
58
+ (enc): RecursiveScriptModule(
59
+ original_name=ConvEncoderImpl
60
+ (conv_head): RecursiveScriptModule(
61
+ original_name=Sequential
62
+ (0): RecursiveScriptModule(original_name=Conv2d)
63
+ (1): RecursiveScriptModule(original_name=ELU)
64
+ (2): RecursiveScriptModule(original_name=Conv2d)
65
+ (3): RecursiveScriptModule(original_name=ELU)
66
+ (4): RecursiveScriptModule(original_name=Conv2d)
67
+ (5): RecursiveScriptModule(original_name=ELU)
68
+ )
69
+ (mlp_layers): RecursiveScriptModule(
70
+ original_name=Sequential
71
+ (0): RecursiveScriptModule(original_name=Linear)
72
+ (1): RecursiveScriptModule(original_name=ELU)
73
+ )
74
+ )
75
+ )
76
+ )
77
+ (core): ModelCoreRNN(
78
+ (core): LSTM(512, 128, num_layers=2)
79
+ )
80
+ (decoder): MlpDecoder(
81
+ (mlp): Identity()
82
+ )
83
+ (critic_linear): Linear(in_features=128, out_features=1, bias=True)
84
+ (action_parameterization): ActionParameterizationDefault(
85
+ (distribution_linear): Linear(in_features=128, out_features=5, bias=True)
86
+ )
87
+ )
88
+ [2025-02-26 18:43:51,065][01151] Using optimizer <class 'torch.optim.adam.Adam'>
89
+ [2025-02-26 18:43:52,838][01151] No checkpoints found
90
+ [2025-02-26 18:43:52,838][01151] Did not load from checkpoint, starting from scratch!
91
+ [2025-02-26 18:43:52,840][01151] Initialized policy 0 weights for model version 0
92
+ [2025-02-26 18:43:52,846][01151] LearnerWorker_p0 finished initialization!
93
+ [2025-02-26 18:43:52,846][01151] Using GPUs [0] for process 0 (actually maps to GPUs [0])
94
+ [2025-02-26 18:43:52,935][01164] RunningMeanStd input shape: (3, 72, 128)
95
+ [2025-02-26 18:43:52,936][01164] RunningMeanStd input shape: (1,)
96
+ [2025-02-26 18:43:52,948][01164] ConvEncoder: input_channels=3
97
+ [2025-02-26 18:43:53,087][01164] Conv encoder output size: 512
98
+ [2025-02-26 18:43:53,087][01164] Policy head output size: 512
99
+ [2025-02-26 18:43:53,131][00031] Inference worker 0-0 is ready!
100
+ [2025-02-26 18:43:53,132][00031] All inference workers are ready! Signal rollout workers to start!
101
+ [2025-02-26 18:43:53,246][01165] Doom resolution: 160x120, resize resolution: (128, 72)
102
+ [2025-02-26 18:43:53,251][01171] Doom resolution: 160x120, resize resolution: (128, 72)
103
+ [2025-02-26 18:43:53,250][01172] Doom resolution: 160x120, resize resolution: (128, 72)
104
+ [2025-02-26 18:43:53,253][01169] Doom resolution: 160x120, resize resolution: (128, 72)
105
+ [2025-02-26 18:43:53,255][01166] Doom resolution: 160x120, resize resolution: (128, 72)
106
+ [2025-02-26 18:43:53,254][01168] Doom resolution: 160x120, resize resolution: (128, 72)
107
+ [2025-02-26 18:43:53,253][01167] Doom resolution: 160x120, resize resolution: (128, 72)
108
+ [2025-02-26 18:43:53,256][01170] Doom resolution: 160x120, resize resolution: (128, 72)
109
+ [2025-02-26 18:43:54,060][01169] Decorrelating experience for 0 frames...
110
+ [2025-02-26 18:43:54,060][01168] Decorrelating experience for 0 frames...
111
+ [2025-02-26 18:43:54,060][01166] Decorrelating experience for 0 frames...
112
+ [2025-02-26 18:43:54,417][01171] Decorrelating experience for 0 frames...
113
+ [2025-02-26 18:43:54,420][01167] Decorrelating experience for 0 frames...
114
+ [2025-02-26 18:43:54,499][01168] Decorrelating experience for 32 frames...
115
+ [2025-02-26 18:43:54,504][01170] Decorrelating experience for 0 frames...
116
+ [2025-02-26 18:43:54,860][01167] Decorrelating experience for 32 frames...
117
+ [2025-02-26 18:43:54,938][01169] Decorrelating experience for 32 frames...
118
+ [2025-02-26 18:43:54,942][01165] Decorrelating experience for 0 frames...
119
+ [2025-02-26 18:43:54,948][01170] Decorrelating experience for 32 frames...
120
+ [2025-02-26 18:43:54,962][01172] Decorrelating experience for 0 frames...
121
+ [2025-02-26 18:43:55,345][01167] Decorrelating experience for 64 frames...
122
+ [2025-02-26 18:43:55,437][01165] Decorrelating experience for 32 frames...
123
+ [2025-02-26 18:43:55,438][01166] Decorrelating experience for 32 frames...
124
+ [2025-02-26 18:43:55,874][01165] Decorrelating experience for 64 frames...
125
+ [2025-02-26 18:43:55,875][01168] Decorrelating experience for 64 frames...
126
+ [2025-02-26 18:43:55,877][01172] Decorrelating experience for 32 frames...
127
+ [2025-02-26 18:43:56,306][01170] Decorrelating experience for 64 frames...
128
+ [2025-02-26 18:43:56,345][01166] Decorrelating experience for 64 frames...
129
+ [2025-02-26 18:43:56,415][01171] Decorrelating experience for 32 frames...
130
+ [2025-02-26 18:43:56,444][01165] Decorrelating experience for 96 frames...
131
+ [2025-02-26 18:43:56,887][01166] Decorrelating experience for 96 frames...
132
+ [2025-02-26 18:43:56,899][00031] Fps is (10 sec: nan, 60 sec: nan, 300 sec: nan). Total num frames: 0. Throughput: 0: nan. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
133
+ [2025-02-26 18:43:56,923][01172] Decorrelating experience for 64 frames...
134
+ [2025-02-26 18:43:57,021][01171] Decorrelating experience for 64 frames...
135
+ [2025-02-26 18:43:57,350][01169] Decorrelating experience for 64 frames...
136
+ [2025-02-26 18:43:57,441][01168] Decorrelating experience for 96 frames...
137
+ [2025-02-26 18:43:57,525][01171] Decorrelating experience for 96 frames...
138
+ [2025-02-26 18:43:58,036][01165] Decorrelating experience for 128 frames...
139
+ [2025-02-26 18:43:58,167][01168] Decorrelating experience for 128 frames...
140
+ [2025-02-26 18:43:58,189][01171] Decorrelating experience for 128 frames...
141
+ [2025-02-26 18:43:58,385][01170] Decorrelating experience for 96 frames...
142
+ [2025-02-26 18:43:58,438][01166] Decorrelating experience for 128 frames...
143
+ [2025-02-26 18:43:58,656][01171] Decorrelating experience for 160 frames...
144
+ [2025-02-26 18:43:58,710][01168] Decorrelating experience for 160 frames...
145
+ [2025-02-26 18:43:58,947][01167] Decorrelating experience for 96 frames...
146
+ [2025-02-26 18:43:59,292][01168] Decorrelating experience for 192 frames...
147
+ [2025-02-26 18:43:59,397][01171] Decorrelating experience for 192 frames...
148
+ [2025-02-26 18:43:59,493][01166] Decorrelating experience for 160 frames...
149
+ [2025-02-26 18:43:59,696][01167] Decorrelating experience for 128 frames...
150
+ [2025-02-26 18:43:59,816][01170] Decorrelating experience for 128 frames...
151
+ [2025-02-26 18:43:59,905][01168] Decorrelating experience for 224 frames...
152
+ [2025-02-26 18:44:00,174][01169] Decorrelating experience for 96 frames...
153
+ [2025-02-26 18:44:00,349][01166] Decorrelating experience for 192 frames...
154
+ [2025-02-26 18:44:00,365][01171] Decorrelating experience for 224 frames...
155
+ [2025-02-26 18:44:00,770][01172] Decorrelating experience for 96 frames...
156
+ [2025-02-26 18:44:00,820][01169] Decorrelating experience for 128 frames...
157
+ [2025-02-26 18:44:01,146][01170] Decorrelating experience for 160 frames...
158
+ [2025-02-26 18:44:01,288][01169] Decorrelating experience for 160 frames...
159
+ [2025-02-26 18:44:01,493][01172] Decorrelating experience for 128 frames...
160
+ [2025-02-26 18:44:01,692][01166] Decorrelating experience for 224 frames...
161
+ [2025-02-26 18:44:01,860][00031] Heartbeat connected on Batcher_0
162
+ [2025-02-26 18:44:01,865][00031] Heartbeat connected on LearnerWorker_p0
163
+ [2025-02-26 18:44:01,891][00031] Heartbeat connected on InferenceWorker_p0-w0
164
+ [2025-02-26 18:44:01,899][00031] Fps is (10 sec: 0.0, 60 sec: 0.0, 300 sec: 0.0). Total num frames: 0. Throughput: 0: 0.0. Samples: 0. Policy #0 lag: (min: -1.0, avg: -1.0, max: -1.0)
165
+ [2025-02-26 18:44:01,901][00031] Heartbeat connected on RolloutWorker_w3
166
+ [2025-02-26 18:44:01,924][00031] Heartbeat connected on RolloutWorker_w6
167
+ [2025-02-26 18:44:02,016][01167] Decorrelating experience for 160 frames...
168
+ [2025-02-26 18:44:02,134][00031] Heartbeat connected on RolloutWorker_w1
169
+ [2025-02-26 18:44:02,221][01169] Decorrelating experience for 192 frames...
170
+ [2025-02-26 18:44:02,340][01172] Decorrelating experience for 160 frames...
171
+ [2025-02-26 18:44:02,344][01170] Decorrelating experience for 192 frames...
172
+ [2025-02-26 18:44:02,742][01165] Decorrelating experience for 160 frames...
173
+ [2025-02-26 18:44:03,560][01167] Decorrelating experience for 192 frames...
174
+ [2025-02-26 18:44:03,620][01169] Decorrelating experience for 224 frames...
175
+ [2025-02-26 18:44:03,931][00031] Heartbeat connected on RolloutWorker_w4
176
+ [2025-02-26 18:44:03,946][01170] Decorrelating experience for 224 frames...
177
+ [2025-02-26 18:44:04,421][00031] Heartbeat connected on RolloutWorker_w5
178
+ [2025-02-26 18:44:04,422][01151] Signal inference workers to stop experience collection...
179
+ [2025-02-26 18:44:04,428][01164] InferenceWorker_p0-w0: stopping experience collection
180
+ [2025-02-26 18:44:04,608][01172] Decorrelating experience for 192 frames...
181
+ [2025-02-26 18:44:04,681][01167] Decorrelating experience for 224 frames...
182
+ [2025-02-26 18:44:04,943][00031] Heartbeat connected on RolloutWorker_w2
183
+ [2025-02-26 18:44:05,039][01165] Decorrelating experience for 192 frames...
184
+ [2025-02-26 18:44:05,180][01172] Decorrelating experience for 224 frames...
185
+ [2025-02-26 18:44:05,366][00031] Heartbeat connected on RolloutWorker_w7
186
+ [2025-02-26 18:44:05,541][01165] Decorrelating experience for 224 frames...
187
+ [2025-02-26 18:44:05,696][00031] Heartbeat connected on RolloutWorker_w0
188
+ [2025-02-26 18:44:06,696][01151] Signal inference workers to resume experience collection...
189
+ [2025-02-26 18:44:06,697][01164] InferenceWorker_p0-w0: resuming experience collection
190
+ [2025-02-26 18:44:06,904][00031] Fps is (10 sec: 409.5, 60 sec: 409.5, 300 sec: 409.5). Total num frames: 4096. Throughput: 0: 269.1. Samples: 2692. Policy #0 lag: (min: 0.0, avg: 0.0, max: 0.0)
191
+ [2025-02-26 18:44:06,908][00031] Avg episode reward: [(0, '2.113')]
192
+ [2025-02-26 18:44:11,009][01164] Updated weights for policy 0, policy_version 10 (0.0164)
193
+ [2025-02-26 18:44:11,899][00031] Fps is (10 sec: 4915.2, 60 sec: 3276.8, 300 sec: 3276.8). Total num frames: 49152. Throughput: 0: 765.6. Samples: 11484. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
194
+ [2025-02-26 18:44:11,901][00031] Avg episode reward: [(0, '3.919')]
195
+ [2025-02-26 18:44:15,535][01164] Updated weights for policy 0, policy_version 20 (0.0015)
196
+ [2025-02-26 18:44:16,899][00031] Fps is (10 sec: 9014.5, 60 sec: 4710.4, 300 sec: 4710.4). Total num frames: 94208. Throughput: 0: 918.8. Samples: 18376. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
197
+ [2025-02-26 18:44:16,901][00031] Avg episode reward: [(0, '4.471')]
198
+ [2025-02-26 18:44:19,548][01164] Updated weights for policy 0, policy_version 30 (0.0018)
199
+ [2025-02-26 18:44:21,899][00031] Fps is (10 sec: 9420.8, 60 sec: 5734.4, 300 sec: 5734.4). Total num frames: 143360. Throughput: 0: 1338.6. Samples: 33464. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
200
+ [2025-02-26 18:44:21,901][00031] Avg episode reward: [(0, '4.376')]
201
+ [2025-02-26 18:44:21,937][01151] Saving new best policy, reward=4.376!
202
+ [2025-02-26 18:44:23,652][01164] Updated weights for policy 0, policy_version 40 (0.0018)
203
+ [2025-02-26 18:44:26,899][00031] Fps is (10 sec: 10240.1, 60 sec: 6553.6, 300 sec: 6553.6). Total num frames: 196608. Throughput: 0: 1610.7. Samples: 48320. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
204
+ [2025-02-26 18:44:26,901][00031] Avg episode reward: [(0, '4.452')]
205
+ [2025-02-26 18:44:26,908][01151] Saving new best policy, reward=4.452!
206
+ [2025-02-26 18:44:27,872][01164] Updated weights for policy 0, policy_version 50 (0.0021)
207
+ [2025-02-26 18:44:31,900][00031] Fps is (10 sec: 9420.6, 60 sec: 6787.6, 300 sec: 6787.6). Total num frames: 237568. Throughput: 0: 1571.5. Samples: 55004. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
208
+ [2025-02-26 18:44:31,901][00031] Avg episode reward: [(0, '4.524')]
209
+ [2025-02-26 18:44:31,902][01151] Saving new best policy, reward=4.524!
210
+ [2025-02-26 18:44:32,593][01164] Updated weights for policy 0, policy_version 60 (0.0017)
211
+ [2025-02-26 18:44:36,608][01164] Updated weights for policy 0, policy_version 70 (0.0015)
212
+ [2025-02-26 18:44:36,900][00031] Fps is (10 sec: 9010.7, 60 sec: 7167.9, 300 sec: 7167.9). Total num frames: 286720. Throughput: 0: 1728.8. Samples: 69152. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
213
+ [2025-02-26 18:44:36,902][00031] Avg episode reward: [(0, '4.481')]
214
+ [2025-02-26 18:44:40,641][01164] Updated weights for policy 0, policy_version 80 (0.0021)
215
+ [2025-02-26 18:44:41,899][00031] Fps is (10 sec: 10240.2, 60 sec: 7554.9, 300 sec: 7554.9). Total num frames: 339968. Throughput: 0: 1873.6. Samples: 84312. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
216
+ [2025-02-26 18:44:41,901][00031] Avg episode reward: [(0, '4.257')]
217
+ [2025-02-26 18:44:44,708][01164] Updated weights for policy 0, policy_version 90 (0.0016)
218
+ [2025-02-26 18:44:46,899][00031] Fps is (10 sec: 10240.7, 60 sec: 7782.4, 300 sec: 7782.4). Total num frames: 389120. Throughput: 0: 2039.6. Samples: 91784. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0)
219
+ [2025-02-26 18:44:46,901][00031] Avg episode reward: [(0, '4.270')]
220
+ [2025-02-26 18:44:48,828][01164] Updated weights for policy 0, policy_version 100 (0.0018)
221
+ [2025-02-26 18:44:51,899][00031] Fps is (10 sec: 9830.3, 60 sec: 7968.6, 300 sec: 7968.6). Total num frames: 438272. Throughput: 0: 2316.8. Samples: 106940. Policy #0 lag: (min: 0.0, avg: 1.5, max: 4.0)
222
+ [2025-02-26 18:44:51,902][00031] Avg episode reward: [(0, '4.383')]
223
+ [2025-02-26 18:44:52,770][01164] Updated weights for policy 0, policy_version 110 (0.0015)
224
+ [2025-02-26 18:44:56,899][00031] Fps is (10 sec: 9830.3, 60 sec: 8123.7, 300 sec: 8123.7). Total num frames: 487424. Throughput: 0: 2454.8. Samples: 121952. Policy #0 lag: (min: 0.0, avg: 1.3, max: 4.0)
225
+ [2025-02-26 18:44:56,901][00031] Avg episode reward: [(0, '4.417')]
226
+ [2025-02-26 18:44:57,108][01164] Updated weights for policy 0, policy_version 120 (0.0016)
227
+ [2025-02-26 18:45:01,103][01164] Updated weights for policy 0, policy_version 130 (0.0015)
228
+ [2025-02-26 18:45:01,900][00031] Fps is (10 sec: 9829.4, 60 sec: 8942.8, 300 sec: 8254.9). Total num frames: 536576. Throughput: 0: 2469.1. Samples: 129488. Policy #0 lag: (min: 0.0, avg: 1.2, max: 4.0)
229
+ [2025-02-26 18:45:01,904][00031] Avg episode reward: [(0, '4.708')]
230
+ [2025-02-26 18:45:01,906][01151] Saving new best policy, reward=4.708!
231
+ [2025-02-26 18:45:05,791][01164] Updated weights for policy 0, policy_version 140 (0.0017)
232
+ [2025-02-26 18:45:06,899][00031] Fps is (10 sec: 9830.5, 60 sec: 9694.5, 300 sec: 8367.6). Total num frames: 585728. Throughput: 0: 2431.6. Samples: 142888. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
233
+ [2025-02-26 18:45:06,901][00031] Avg episode reward: [(0, '4.626')]
234
+ [2025-02-26 18:45:09,980][01164] Updated weights for policy 0, policy_version 150 (0.0019)
235
+ [2025-02-26 18:45:11,899][00031] Fps is (10 sec: 9421.8, 60 sec: 9693.9, 300 sec: 8410.5). Total num frames: 630784. Throughput: 0: 2432.3. Samples: 157772. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0)
236
+ [2025-02-26 18:45:11,901][00031] Avg episode reward: [(0, '4.413')]
237
+ [2025-02-26 18:45:14,028][01164] Updated weights for policy 0, policy_version 160 (0.0017)
238
+ [2025-02-26 18:45:16,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9830.4, 300 sec: 8550.4). Total num frames: 684032. Throughput: 0: 2447.2. Samples: 165128. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
239
+ [2025-02-26 18:45:16,901][00031] Avg episode reward: [(0, '4.488')]
240
+ [2025-02-26 18:45:18,162][01164] Updated weights for policy 0, policy_version 170 (0.0019)
241
+ [2025-02-26 18:45:21,899][00031] Fps is (10 sec: 10240.0, 60 sec: 9830.4, 300 sec: 8625.7). Total num frames: 733184. Throughput: 0: 2469.4. Samples: 180272. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0)
242
+ [2025-02-26 18:45:21,901][00031] Avg episode reward: [(0, '4.576')]
243
+ [2025-02-26 18:45:22,088][01164] Updated weights for policy 0, policy_version 180 (0.0016)
244
+ [2025-02-26 18:45:26,243][01164] Updated weights for policy 0, policy_version 190 (0.0018)
245
+ [2025-02-26 18:45:26,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9762.1, 300 sec: 8692.6). Total num frames: 782336. Throughput: 0: 2467.1. Samples: 195332. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0)
246
+ [2025-02-26 18:45:26,901][00031] Avg episode reward: [(0, '4.477')]
247
+ [2025-02-26 18:45:30,247][01164] Updated weights for policy 0, policy_version 200 (0.0021)
248
+ [2025-02-26 18:45:31,899][00031] Fps is (10 sec: 10240.0, 60 sec: 9967.0, 300 sec: 8795.6). Total num frames: 835584. Throughput: 0: 2470.3. Samples: 202948. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
249
+ [2025-02-26 18:45:31,901][00031] Avg episode reward: [(0, '4.327')]
250
+ [2025-02-26 18:45:34,780][01164] Updated weights for policy 0, policy_version 210 (0.0020)
251
+ [2025-02-26 18:45:36,900][00031] Fps is (10 sec: 9420.3, 60 sec: 9830.4, 300 sec: 8765.4). Total num frames: 876544. Throughput: 0: 2438.5. Samples: 216672. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
252
+ [2025-02-26 18:45:36,902][00031] Avg episode reward: [(0, '4.658')]
253
+ [2025-02-26 18:45:36,909][01151] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000214_876544.pth...
254
+ [2025-02-26 18:45:39,200][01164] Updated weights for policy 0, policy_version 220 (0.0021)
255
+ [2025-02-26 18:45:41,899][00031] Fps is (10 sec: 9011.2, 60 sec: 9762.1, 300 sec: 8816.2). Total num frames: 925696. Throughput: 0: 2422.8. Samples: 230980. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
256
+ [2025-02-26 18:45:41,904][00031] Avg episode reward: [(0, '4.397')]
257
+ [2025-02-26 18:45:43,391][01164] Updated weights for policy 0, policy_version 230 (0.0016)
258
+ [2025-02-26 18:45:46,899][00031] Fps is (10 sec: 9830.9, 60 sec: 9762.1, 300 sec: 8862.3). Total num frames: 974848. Throughput: 0: 2415.9. Samples: 238200. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0)
259
+ [2025-02-26 18:45:46,902][00031] Avg episode reward: [(0, '4.708')]
260
+ [2025-02-26 18:45:47,705][01164] Updated weights for policy 0, policy_version 240 (0.0018)
261
+ [2025-02-26 18:45:51,715][01164] Updated weights for policy 0, policy_version 250 (0.0018)
262
+ [2025-02-26 18:45:51,899][00031] Fps is (10 sec: 9830.3, 60 sec: 9762.1, 300 sec: 8904.4). Total num frames: 1024000. Throughput: 0: 2446.2. Samples: 252968. Policy #0 lag: (min: 0.0, avg: 1.4, max: 2.0)
263
+ [2025-02-26 18:45:51,903][00031] Avg episode reward: [(0, '4.491')]
264
+ [2025-02-26 18:45:55,982][01164] Updated weights for policy 0, policy_version 260 (0.0018)
265
+ [2025-02-26 18:45:56,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9762.1, 300 sec: 8942.9). Total num frames: 1073152. Throughput: 0: 2445.4. Samples: 267816. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0)
266
+ [2025-02-26 18:45:56,901][00031] Avg episode reward: [(0, '4.508')]
267
+ [2025-02-26 18:46:00,050][01164] Updated weights for policy 0, policy_version 270 (0.0017)
268
+ [2025-02-26 18:46:01,900][00031] Fps is (10 sec: 9830.1, 60 sec: 9762.2, 300 sec: 8978.4). Total num frames: 1122304. Throughput: 0: 2449.8. Samples: 275368. Policy #0 lag: (min: 0.0, avg: 1.2, max: 4.0)
269
+ [2025-02-26 18:46:01,902][00031] Avg episode reward: [(0, '4.604')]
270
+ [2025-02-26 18:46:04,134][01164] Updated weights for policy 0, policy_version 280 (0.0016)
271
+ [2025-02-26 18:46:06,899][00031] Fps is (10 sec: 9830.5, 60 sec: 9762.1, 300 sec: 9011.2). Total num frames: 1171456. Throughput: 0: 2447.2. Samples: 290396. Policy #0 lag: (min: 0.0, avg: 0.7, max: 3.0)
272
+ [2025-02-26 18:46:06,901][00031] Avg episode reward: [(0, '4.539')]
273
+ [2025-02-26 18:46:08,858][01164] Updated weights for policy 0, policy_version 290 (0.0018)
274
+ [2025-02-26 18:46:11,899][00031] Fps is (10 sec: 9421.1, 60 sec: 9762.1, 300 sec: 9011.2). Total num frames: 1216512. Throughput: 0: 2414.1. Samples: 303968. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
275
+ [2025-02-26 18:46:11,901][00031] Avg episode reward: [(0, '4.492')]
276
+ [2025-02-26 18:46:12,814][01164] Updated weights for policy 0, policy_version 300 (0.0017)
277
+ [2025-02-26 18:46:16,900][00031] Fps is (10 sec: 9420.6, 60 sec: 9693.8, 300 sec: 9040.4). Total num frames: 1265664. Throughput: 0: 2408.6. Samples: 311336. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
278
+ [2025-02-26 18:46:16,901][00031] Avg episode reward: [(0, '4.301')]
279
+ [2025-02-26 18:46:16,989][01164] Updated weights for policy 0, policy_version 310 (0.0015)
280
+ [2025-02-26 18:46:20,964][01164] Updated weights for policy 0, policy_version 320 (0.0018)
281
+ [2025-02-26 18:46:21,899][00031] Fps is (10 sec: 9830.5, 60 sec: 9693.9, 300 sec: 9067.7). Total num frames: 1314816. Throughput: 0: 2439.9. Samples: 326464. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
282
+ [2025-02-26 18:46:21,901][00031] Avg episode reward: [(0, '4.431')]
283
+ [2025-02-26 18:46:25,238][01164] Updated weights for policy 0, policy_version 330 (0.0019)
284
+ [2025-02-26 18:46:26,899][00031] Fps is (10 sec: 9830.6, 60 sec: 9693.9, 300 sec: 9093.1). Total num frames: 1363968. Throughput: 0: 2454.0. Samples: 341412. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
285
+ [2025-02-26 18:46:26,901][00031] Avg episode reward: [(0, '4.508')]
286
+ [2025-02-26 18:46:29,381][01164] Updated weights for policy 0, policy_version 340 (0.0016)
287
+ [2025-02-26 18:46:31,899][00031] Fps is (10 sec: 10239.9, 60 sec: 9693.9, 300 sec: 9143.3). Total num frames: 1417216. Throughput: 0: 2458.5. Samples: 348832. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0)
288
+ [2025-02-26 18:46:31,901][00031] Avg episode reward: [(0, '4.684')]
289
+ [2025-02-26 18:46:33,496][01164] Updated weights for policy 0, policy_version 350 (0.0018)
290
+ [2025-02-26 18:46:36,899][00031] Fps is (10 sec: 10240.0, 60 sec: 9830.5, 300 sec: 9164.8). Total num frames: 1466368. Throughput: 0: 2458.6. Samples: 363604. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
291
+ [2025-02-26 18:46:36,901][00031] Avg episode reward: [(0, '4.505')]
292
+ [2025-02-26 18:46:37,719][01164] Updated weights for policy 0, policy_version 360 (0.0017)
293
+ [2025-02-26 18:46:41,899][00031] Fps is (10 sec: 9420.7, 60 sec: 9762.1, 300 sec: 9160.1). Total num frames: 1511424. Throughput: 0: 2426.9. Samples: 377028. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
294
+ [2025-02-26 18:46:41,903][00031] Avg episode reward: [(0, '4.740')]
295
+ [2025-02-26 18:46:41,906][01151] Saving new best policy, reward=4.740!
296
+ [2025-02-26 18:46:42,361][01164] Updated weights for policy 0, policy_version 370 (0.0015)
297
+ [2025-02-26 18:46:46,642][01164] Updated weights for policy 0, policy_version 380 (0.0019)
298
+ [2025-02-26 18:46:46,899][00031] Fps is (10 sec: 9011.1, 60 sec: 9693.9, 300 sec: 9155.8). Total num frames: 1556480. Throughput: 0: 2417.5. Samples: 384156. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
299
+ [2025-02-26 18:46:46,901][00031] Avg episode reward: [(0, '4.899')]
300
+ [2025-02-26 18:46:46,912][01151] Saving new best policy, reward=4.899!
301
+ [2025-02-26 18:46:50,725][01164] Updated weights for policy 0, policy_version 390 (0.0016)
302
+ [2025-02-26 18:46:51,899][00031] Fps is (10 sec: 9420.9, 60 sec: 9693.9, 300 sec: 9175.0). Total num frames: 1605632. Throughput: 0: 2413.8. Samples: 399016. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
303
+ [2025-02-26 18:46:51,901][00031] Avg episode reward: [(0, '4.655')]
304
+ [2025-02-26 18:46:54,924][01164] Updated weights for policy 0, policy_version 400 (0.0017)
305
+ [2025-02-26 18:46:56,899][00031] Fps is (10 sec: 10240.1, 60 sec: 9762.1, 300 sec: 9216.0). Total num frames: 1658880. Throughput: 0: 2440.0. Samples: 413768. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
306
+ [2025-02-26 18:46:56,902][00031] Avg episode reward: [(0, '4.877')]
307
+ [2025-02-26 18:46:59,082][01164] Updated weights for policy 0, policy_version 410 (0.0015)
308
+ [2025-02-26 18:47:01,899][00031] Fps is (10 sec: 10240.1, 60 sec: 9762.2, 300 sec: 9232.6). Total num frames: 1708032. Throughput: 0: 2442.9. Samples: 421268. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
309
+ [2025-02-26 18:47:01,903][00031] Avg episode reward: [(0, '4.948')]
310
+ [2025-02-26 18:47:01,905][01151] Saving new best policy, reward=4.948!
311
+ [2025-02-26 18:47:03,193][01164] Updated weights for policy 0, policy_version 420 (0.0018)
312
+ [2025-02-26 18:47:06,899][00031] Fps is (10 sec: 9420.8, 60 sec: 9693.9, 300 sec: 9226.8). Total num frames: 1753088. Throughput: 0: 2435.4. Samples: 436056. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
313
+ [2025-02-26 18:47:06,902][00031] Avg episode reward: [(0, '4.642')]
314
+ [2025-02-26 18:47:07,296][01164] Updated weights for policy 0, policy_version 430 (0.0021)
315
+ [2025-02-26 18:47:11,784][01164] Updated weights for policy 0, policy_version 440 (0.0017)
316
+ [2025-02-26 18:47:11,899][00031] Fps is (10 sec: 9420.8, 60 sec: 9762.1, 300 sec: 9242.3). Total num frames: 1802240. Throughput: 0: 2424.4. Samples: 450512. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
317
+ [2025-02-26 18:47:11,901][00031] Avg episode reward: [(0, '4.935')]
318
+ [2025-02-26 18:47:16,285][01164] Updated weights for policy 0, policy_version 450 (0.0026)
319
+ [2025-02-26 18:47:16,899][00031] Fps is (10 sec: 9420.8, 60 sec: 9693.9, 300 sec: 9236.5). Total num frames: 1847296. Throughput: 0: 2396.3. Samples: 456664. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
320
+ [2025-02-26 18:47:16,901][00031] Avg episode reward: [(0, '4.896')]
321
+ [2025-02-26 18:47:20,419][01164] Updated weights for policy 0, policy_version 460 (0.0015)
322
+ [2025-02-26 18:47:21,899][00031] Fps is (10 sec: 9420.8, 60 sec: 9693.9, 300 sec: 9251.0). Total num frames: 1896448. Throughput: 0: 2398.2. Samples: 471524. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0)
323
+ [2025-02-26 18:47:21,901][00031] Avg episode reward: [(0, '4.634')]
324
+ [2025-02-26 18:47:24,499][01164] Updated weights for policy 0, policy_version 470 (0.0016)
325
+ [2025-02-26 18:47:26,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9693.9, 300 sec: 9264.8). Total num frames: 1945600. Throughput: 0: 2431.1. Samples: 486428. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
326
+ [2025-02-26 18:47:26,901][00031] Avg episode reward: [(0, '5.011')]
327
+ [2025-02-26 18:47:26,907][01151] Saving new best policy, reward=5.011!
328
+ [2025-02-26 18:47:28,769][01164] Updated weights for policy 0, policy_version 480 (0.0016)
329
+ [2025-02-26 18:47:31,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9277.9). Total num frames: 1994752. Throughput: 0: 2434.9. Samples: 493724. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
330
+ [2025-02-26 18:47:31,901][00031] Avg episode reward: [(0, '4.912')]
331
+ [2025-02-26 18:47:32,738][01164] Updated weights for policy 0, policy_version 490 (0.0019)
332
+ [2025-02-26 18:47:36,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9290.5). Total num frames: 2043904. Throughput: 0: 2435.8. Samples: 508628. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
333
+ [2025-02-26 18:47:36,901][00031] Avg episode reward: [(0, '5.212')]
334
+ [2025-02-26 18:47:36,909][01151] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000499_2043904.pth...
335
+ [2025-02-26 18:47:36,969][01151] Saving new best policy, reward=5.212!
336
+ [2025-02-26 18:47:37,106][01164] Updated weights for policy 0, policy_version 500 (0.0016)
337
+ [2025-02-26 18:47:40,976][01164] Updated weights for policy 0, policy_version 510 (0.0016)
338
+ [2025-02-26 18:47:41,899][00031] Fps is (10 sec: 10240.0, 60 sec: 9762.2, 300 sec: 9320.7). Total num frames: 2097152. Throughput: 0: 2439.6. Samples: 523548. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
339
+ [2025-02-26 18:47:41,900][00031] Avg episode reward: [(0, '5.423')]
340
+ [2025-02-26 18:47:41,902][01151] Saving new best policy, reward=5.423!
341
+ [2025-02-26 18:47:45,923][01164] Updated weights for policy 0, policy_version 520 (0.0018)
342
+ [2025-02-26 18:47:46,899][00031] Fps is (10 sec: 9420.6, 60 sec: 9693.8, 300 sec: 9296.1). Total num frames: 2138112. Throughput: 0: 2421.1. Samples: 530220. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
343
+ [2025-02-26 18:47:46,901][00031] Avg episode reward: [(0, '5.034')]
344
+ [2025-02-26 18:47:50,029][01164] Updated weights for policy 0, policy_version 530 (0.0020)
345
+ [2025-02-26 18:47:51,899][00031] Fps is (10 sec: 8601.6, 60 sec: 9625.6, 300 sec: 9290.1). Total num frames: 2183168. Throughput: 0: 2395.3. Samples: 543844. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
346
+ [2025-02-26 18:47:51,901][00031] Avg episode reward: [(0, '5.208')]
347
+ [2025-02-26 18:47:54,382][01164] Updated weights for policy 0, policy_version 540 (0.0019)
348
+ [2025-02-26 18:47:56,899][00031] Fps is (10 sec: 9830.6, 60 sec: 9625.6, 300 sec: 9318.4). Total num frames: 2236416. Throughput: 0: 2398.5. Samples: 558444. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
349
+ [2025-02-26 18:47:56,901][00031] Avg episode reward: [(0, '5.176')]
350
+ [2025-02-26 18:47:58,496][01164] Updated weights for policy 0, policy_version 550 (0.0015)
351
+ [2025-02-26 18:48:01,899][00031] Fps is (10 sec: 10239.9, 60 sec: 9625.6, 300 sec: 9328.8). Total num frames: 2285568. Throughput: 0: 2429.5. Samples: 565992. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
352
+ [2025-02-26 18:48:01,901][00031] Avg episode reward: [(0, '5.512')]
353
+ [2025-02-26 18:48:01,903][01151] Saving new best policy, reward=5.512!
354
+ [2025-02-26 18:48:02,751][01164] Updated weights for policy 0, policy_version 560 (0.0025)
355
+ [2025-02-26 18:48:06,706][01164] Updated weights for policy 0, policy_version 570 (0.0016)
356
+ [2025-02-26 18:48:06,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9693.9, 300 sec: 9338.9). Total num frames: 2334720. Throughput: 0: 2426.1. Samples: 580700. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
357
+ [2025-02-26 18:48:06,901][00031] Avg episode reward: [(0, '5.303')]
358
+ [2025-02-26 18:48:11,047][01164] Updated weights for policy 0, policy_version 580 (0.0017)
359
+ [2025-02-26 18:48:11,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9693.9, 300 sec: 9348.5). Total num frames: 2383872. Throughput: 0: 2423.6. Samples: 595492. Policy #0 lag: (min: 0.0, avg: 0.6, max: 2.0)
360
+ [2025-02-26 18:48:11,904][00031] Avg episode reward: [(0, '5.573')]
361
+ [2025-02-26 18:48:11,907][01151] Saving new best policy, reward=5.573!
362
+ [2025-02-26 18:48:15,010][01164] Updated weights for policy 0, policy_version 590 (0.0019)
363
+ [2025-02-26 18:48:16,899][00031] Fps is (10 sec: 9420.8, 60 sec: 9693.9, 300 sec: 9342.0). Total num frames: 2428928. Throughput: 0: 2424.7. Samples: 602836. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
364
+ [2025-02-26 18:48:16,903][00031] Avg episode reward: [(0, '5.203')]
365
+ [2025-02-26 18:48:19,769][01164] Updated weights for policy 0, policy_version 600 (0.0016)
366
+ [2025-02-26 18:48:21,899][00031] Fps is (10 sec: 9011.3, 60 sec: 9625.6, 300 sec: 9335.8). Total num frames: 2473984. Throughput: 0: 2387.0. Samples: 616044. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
367
+ [2025-02-26 18:48:21,901][00031] Avg episode reward: [(0, '5.402')]
368
+ [2025-02-26 18:48:24,121][01164] Updated weights for policy 0, policy_version 610 (0.0018)
369
+ [2025-02-26 18:48:26,900][00031] Fps is (10 sec: 9420.6, 60 sec: 9625.6, 300 sec: 9344.9). Total num frames: 2523136. Throughput: 0: 2382.3. Samples: 630752. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
370
+ [2025-02-26 18:48:26,901][00031] Avg episode reward: [(0, '5.474')]
371
+ [2025-02-26 18:48:28,247][01164] Updated weights for policy 0, policy_version 620 (0.0017)
372
+ [2025-02-26 18:48:31,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9353.8). Total num frames: 2572288. Throughput: 0: 2398.6. Samples: 638156. Policy #0 lag: (min: 0.0, avg: 1.0, max: 4.0)
373
+ [2025-02-26 18:48:31,901][00031] Avg episode reward: [(0, '5.841')]
374
+ [2025-02-26 18:48:31,904][01151] Saving new best policy, reward=5.841!
375
+ [2025-02-26 18:48:32,503][01164] Updated weights for policy 0, policy_version 630 (0.0021)
376
+ [2025-02-26 18:48:36,462][01164] Updated weights for policy 0, policy_version 640 (0.0017)
377
+ [2025-02-26 18:48:36,899][00031] Fps is (10 sec: 9830.7, 60 sec: 9625.6, 300 sec: 9362.3). Total num frames: 2621440. Throughput: 0: 2425.8. Samples: 653004. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0)
378
+ [2025-02-26 18:48:36,900][00031] Avg episode reward: [(0, '6.122')]
379
+ [2025-02-26 18:48:36,908][01151] Saving new best policy, reward=6.122!
380
+ [2025-02-26 18:48:40,510][01164] Updated weights for policy 0, policy_version 650 (0.0022)
381
+ [2025-02-26 18:48:41,899][00031] Fps is (10 sec: 10240.1, 60 sec: 9625.6, 300 sec: 9384.9). Total num frames: 2674688. Throughput: 0: 2430.9. Samples: 667836. Policy #0 lag: (min: 0.0, avg: 0.8, max: 3.0)
382
+ [2025-02-26 18:48:41,900][00031] Avg episode reward: [(0, '5.253')]
383
+ [2025-02-26 18:48:44,899][01164] Updated weights for policy 0, policy_version 660 (0.0019)
384
+ [2025-02-26 18:48:46,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9693.9, 300 sec: 9378.4). Total num frames: 2719744. Throughput: 0: 2425.1. Samples: 675120. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0)
385
+ [2025-02-26 18:48:46,901][00031] Avg episode reward: [(0, '5.196')]
386
+ [2025-02-26 18:48:49,143][01164] Updated weights for policy 0, policy_version 670 (0.0016)
387
+ [2025-02-26 18:48:51,899][00031] Fps is (10 sec: 9011.1, 60 sec: 9693.9, 300 sec: 9372.2). Total num frames: 2764800. Throughput: 0: 2405.4. Samples: 688944. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0)
388
+ [2025-02-26 18:48:51,900][00031] Avg episode reward: [(0, '5.378')]
389
+ [2025-02-26 18:48:53,770][01164] Updated weights for policy 0, policy_version 680 (0.0017)
390
+ [2025-02-26 18:48:56,899][00031] Fps is (10 sec: 9420.8, 60 sec: 9625.6, 300 sec: 9538.8). Total num frames: 2813952. Throughput: 0: 2397.2. Samples: 703364. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
391
+ [2025-02-26 18:48:56,901][00031] Avg episode reward: [(0, '5.623')]
392
+ [2025-02-26 18:48:57,877][01164] Updated weights for policy 0, policy_version 690 (0.0018)
393
+ [2025-02-26 18:49:01,900][00031] Fps is (10 sec: 9830.1, 60 sec: 9625.6, 300 sec: 9691.7). Total num frames: 2863104. Throughput: 0: 2390.8. Samples: 710424. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
394
+ [2025-02-26 18:49:01,902][00031] Avg episode reward: [(0, '5.594')]
395
+ [2025-02-26 18:49:02,196][01164] Updated weights for policy 0, policy_version 700 (0.0020)
396
+ [2025-02-26 18:49:06,315][01164] Updated weights for policy 0, policy_version 710 (0.0019)
397
+ [2025-02-26 18:49:06,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9625.6, 300 sec: 9705.4). Total num frames: 2912256. Throughput: 0: 2425.8. Samples: 725204. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
398
+ [2025-02-26 18:49:06,901][00031] Avg episode reward: [(0, '5.313')]
399
+ [2025-02-26 18:49:10,384][01164] Updated weights for policy 0, policy_version 720 (0.0023)
400
+ [2025-02-26 18:49:11,899][00031] Fps is (10 sec: 9830.7, 60 sec: 9625.6, 300 sec: 9719.3). Total num frames: 2961408. Throughput: 0: 2430.9. Samples: 740144. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
401
+ [2025-02-26 18:49:11,901][00031] Avg episode reward: [(0, '5.271')]
402
+ [2025-02-26 18:49:14,485][01164] Updated weights for policy 0, policy_version 730 (0.0017)
403
+ [2025-02-26 18:49:16,900][00031] Fps is (10 sec: 9830.2, 60 sec: 9693.8, 300 sec: 9719.3). Total num frames: 3010560. Throughput: 0: 2430.0. Samples: 747508. Policy #0 lag: (min: 0.0, avg: 1.1, max: 2.0)
404
+ [2025-02-26 18:49:16,901][00031] Avg episode reward: [(0, '5.705')]
405
+ [2025-02-26 18:49:18,739][01164] Updated weights for policy 0, policy_version 740 (0.0022)
406
+ [2025-02-26 18:49:21,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9762.1, 300 sec: 9705.4). Total num frames: 3059712. Throughput: 0: 2434.8. Samples: 762568. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
407
+ [2025-02-26 18:49:21,901][00031] Avg episode reward: [(0, '5.736')]
408
+ [2025-02-26 18:49:23,346][01164] Updated weights for policy 0, policy_version 750 (0.0019)
409
+ [2025-02-26 18:49:26,899][00031] Fps is (10 sec: 9421.0, 60 sec: 9693.9, 300 sec: 9719.3). Total num frames: 3104768. Throughput: 0: 2404.7. Samples: 776048. Policy #0 lag: (min: 0.0, avg: 1.1, max: 4.0)
410
+ [2025-02-26 18:49:26,901][00031] Avg episode reward: [(0, '5.284')]
411
+ [2025-02-26 18:49:27,587][01164] Updated weights for policy 0, policy_version 760 (0.0020)
412
+ [2025-02-26 18:49:31,698][01164] Updated weights for policy 0, policy_version 770 (0.0020)
413
+ [2025-02-26 18:49:31,899][00031] Fps is (10 sec: 9420.9, 60 sec: 9693.9, 300 sec: 9719.3). Total num frames: 3153920. Throughput: 0: 2406.9. Samples: 783432. Policy #0 lag: (min: 0.0, avg: 0.8, max: 2.0)
414
+ [2025-02-26 18:49:31,900][00031] Avg episode reward: [(0, '5.964')]
415
+ [2025-02-26 18:49:35,728][01164] Updated weights for policy 0, policy_version 780 (0.0017)
416
+ [2025-02-26 18:49:36,899][00031] Fps is (10 sec: 9830.3, 60 sec: 9693.9, 300 sec: 9705.4). Total num frames: 3203072. Throughput: 0: 2430.9. Samples: 798336. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0)
417
+ [2025-02-26 18:49:36,904][00031] Avg episode reward: [(0, '5.459')]
418
+ [2025-02-26 18:49:36,924][01151] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000783_3207168.pth...
419
+ [2025-02-26 18:49:36,984][01151] Removing /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000214_876544.pth
420
+ [2025-02-26 18:49:39,852][01164] Updated weights for policy 0, policy_version 790 (0.0015)
421
+ [2025-02-26 18:49:41,899][00031] Fps is (10 sec: 10240.0, 60 sec: 9693.9, 300 sec: 9719.3). Total num frames: 3256320. Throughput: 0: 2445.7. Samples: 813420. Policy #0 lag: (min: 0.0, avg: 0.9, max: 2.0)
422
+ [2025-02-26 18:49:41,901][00031] Avg episode reward: [(0, '5.405')]
423
+ [2025-02-26 18:49:43,947][01164] Updated weights for policy 0, policy_version 800 (0.0018)
424
+ [2025-02-26 18:49:46,899][00031] Fps is (10 sec: 10240.0, 60 sec: 9762.1, 300 sec: 9719.3). Total num frames: 3305472. Throughput: 0: 2454.1. Samples: 820856. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
425
+ [2025-02-26 18:49:46,900][00031] Avg episode reward: [(0, '6.214')]
426
+ [2025-02-26 18:49:46,910][01151] Saving new best policy, reward=6.214!
427
+ [2025-02-26 18:49:47,981][01164] Updated weights for policy 0, policy_version 810 (0.0017)
428
+ [2025-02-26 18:49:51,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9830.4, 300 sec: 9719.3). Total num frames: 3354624. Throughput: 0: 2457.9. Samples: 835808. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
429
+ [2025-02-26 18:49:51,901][00031] Avg episode reward: [(0, '6.287')]
430
+ [2025-02-26 18:49:51,904][01151] Saving new best policy, reward=6.287!
431
+ [2025-02-26 18:49:52,189][01164] Updated weights for policy 0, policy_version 820 (0.0019)
432
+ [2025-02-26 18:49:56,899][00031] Fps is (10 sec: 9011.2, 60 sec: 9693.9, 300 sec: 9691.6). Total num frames: 3395584. Throughput: 0: 2421.7. Samples: 849120. Policy #0 lag: (min: 0.0, avg: 0.7, max: 2.0)
433
+ [2025-02-26 18:49:56,902][00031] Avg episode reward: [(0, '5.601')]
434
+ [2025-02-26 18:49:56,980][01164] Updated weights for policy 0, policy_version 830 (0.0020)
435
+ [2025-02-26 18:50:01,097][01164] Updated weights for policy 0, policy_version 840 (0.0016)
436
+ [2025-02-26 18:50:01,899][00031] Fps is (10 sec: 9420.8, 60 sec: 9762.2, 300 sec: 9705.4). Total num frames: 3448832. Throughput: 0: 2425.7. Samples: 856664. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
437
+ [2025-02-26 18:50:01,901][00031] Avg episode reward: [(0, '6.318')]
438
+ [2025-02-26 18:50:01,903][01151] Saving new best policy, reward=6.318!
439
+ [2025-02-26 18:50:05,179][01164] Updated weights for policy 0, policy_version 850 (0.0018)
440
+ [2025-02-26 18:50:06,899][00031] Fps is (10 sec: 10240.0, 60 sec: 9762.1, 300 sec: 9719.3). Total num frames: 3497984. Throughput: 0: 2420.1. Samples: 871472. Policy #0 lag: (min: 0.0, avg: 1.2, max: 3.0)
441
+ [2025-02-26 18:50:06,901][00031] Avg episode reward: [(0, '6.070')]
442
+ [2025-02-26 18:50:09,164][01164] Updated weights for policy 0, policy_version 860 (0.0020)
443
+ [2025-02-26 18:50:11,899][00031] Fps is (10 sec: 9830.3, 60 sec: 9762.1, 300 sec: 9705.4). Total num frames: 3547136. Throughput: 0: 2455.6. Samples: 886552. Policy #0 lag: (min: 0.0, avg: 1.3, max: 3.0)
444
+ [2025-02-26 18:50:11,901][00031] Avg episode reward: [(0, '6.180')]
445
+ [2025-02-26 18:50:13,445][01164] Updated weights for policy 0, policy_version 870 (0.0019)
446
+ [2025-02-26 18:50:16,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9762.2, 300 sec: 9705.4). Total num frames: 3596288. Throughput: 0: 2454.7. Samples: 893892. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
447
+ [2025-02-26 18:50:16,901][00031] Avg episode reward: [(0, '5.804')]
448
+ [2025-02-26 18:50:17,441][01164] Updated weights for policy 0, policy_version 880 (0.0018)
449
+ [2025-02-26 18:50:21,578][01164] Updated weights for policy 0, policy_version 890 (0.0020)
450
+ [2025-02-26 18:50:21,901][00031] Fps is (10 sec: 9829.0, 60 sec: 9761.9, 300 sec: 9705.4). Total num frames: 3645440. Throughput: 0: 2456.1. Samples: 908864. Policy #0 lag: (min: 0.0, avg: 0.9, max: 3.0)
451
+ [2025-02-26 18:50:21,902][00031] Avg episode reward: [(0, '5.803')]
452
+ [2025-02-26 18:50:25,928][01164] Updated weights for policy 0, policy_version 900 (0.0019)
453
+ [2025-02-26 18:50:26,899][00031] Fps is (10 sec: 9420.8, 60 sec: 9762.1, 300 sec: 9677.7). Total num frames: 3690496. Throughput: 0: 2447.6. Samples: 923564. Policy #0 lag: (min: 0.0, avg: 1.0, max: 3.0)
454
+ [2025-02-26 18:50:26,905][00031] Avg episode reward: [(0, '5.848')]
455
+ [2025-02-26 18:50:30,505][01164] Updated weights for policy 0, policy_version 910 (0.0018)
456
+ [2025-02-26 18:50:31,899][00031] Fps is (10 sec: 9422.2, 60 sec: 9762.1, 300 sec: 9705.5). Total num frames: 3739648. Throughput: 0: 2416.1. Samples: 929580. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
457
+ [2025-02-26 18:50:31,901][00031] Avg episode reward: [(0, '5.100')]
458
+ [2025-02-26 18:50:34,811][01164] Updated weights for policy 0, policy_version 920 (0.0016)
459
+ [2025-02-26 18:50:36,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9762.1, 300 sec: 9705.4). Total num frames: 3788800. Throughput: 0: 2409.0. Samples: 944212. Policy #0 lag: (min: 0.0, avg: 1.4, max: 3.0)
460
+ [2025-02-26 18:50:36,901][00031] Avg episode reward: [(0, '5.964')]
461
+ [2025-02-26 18:50:38,729][01164] Updated weights for policy 0, policy_version 930 (0.0016)
462
+ [2025-02-26 18:50:41,902][00031] Fps is (10 sec: 9828.0, 60 sec: 9693.5, 300 sec: 9705.4). Total num frames: 3837952. Throughput: 0: 2445.7. Samples: 959184. Policy #0 lag: (min: 0.0, avg: 1.3, max: 4.0)
463
+ [2025-02-26 18:50:41,904][00031] Avg episode reward: [(0, '5.876')]
464
+ [2025-02-26 18:50:43,009][01164] Updated weights for policy 0, policy_version 940 (0.0019)
465
+ [2025-02-26 18:50:46,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9693.9, 300 sec: 9705.4). Total num frames: 3887104. Throughput: 0: 2441.2. Samples: 966516. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
466
+ [2025-02-26 18:50:46,901][00031] Avg episode reward: [(0, '5.769')]
467
+ [2025-02-26 18:50:46,985][01164] Updated weights for policy 0, policy_version 950 (0.0017)
468
+ [2025-02-26 18:50:51,313][01164] Updated weights for policy 0, policy_version 960 (0.0016)
469
+ [2025-02-26 18:50:51,899][00031] Fps is (10 sec: 9832.9, 60 sec: 9693.9, 300 sec: 9705.4). Total num frames: 3936256. Throughput: 0: 2445.7. Samples: 981528. Policy #0 lag: (min: 0.0, avg: 1.0, max: 2.0)
470
+ [2025-02-26 18:50:51,901][00031] Avg episode reward: [(0, '6.021')]
471
+ [2025-02-26 18:50:55,355][01164] Updated weights for policy 0, policy_version 970 (0.0017)
472
+ [2025-02-26 18:50:56,899][00031] Fps is (10 sec: 9830.4, 60 sec: 9830.4, 300 sec: 9705.4). Total num frames: 3985408. Throughput: 0: 2438.6. Samples: 996288. Policy #0 lag: (min: 0.0, avg: 1.1, max: 3.0)
473
+ [2025-02-26 18:50:56,901][00031] Avg episode reward: [(0, '5.460')]
474
+ [2025-02-26 18:50:58,672][01151] Stopping Batcher_0...
475
+ [2025-02-26 18:50:58,672][01151] Loop batcher_evt_loop terminating...
476
+ [2025-02-26 18:50:58,673][01151] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
477
+ [2025-02-26 18:50:58,672][00031] Component Batcher_0 stopped!
478
+ [2025-02-26 18:50:58,707][01164] Weights refcount: 2 0
479
+ [2025-02-26 18:50:58,714][01164] Stopping InferenceWorker_p0-w0...
480
+ [2025-02-26 18:50:58,715][01164] Loop inference_proc0-0_evt_loop terminating...
481
+ [2025-02-26 18:50:58,715][00031] Component InferenceWorker_p0-w0 stopped!
482
+ [2025-02-26 18:50:58,733][01151] Removing /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000499_2043904.pth
483
+ [2025-02-26 18:50:58,740][01151] Saving /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
484
+ [2025-02-26 18:50:58,839][01151] Stopping LearnerWorker_p0...
485
+ [2025-02-26 18:50:58,840][01151] Loop learner_proc0_evt_loop terminating...
486
+ [2025-02-26 18:50:58,840][00031] Component LearnerWorker_p0 stopped!
487
+ [2025-02-26 18:50:58,970][00031] Component RolloutWorker_w6 stopped!
488
+ [2025-02-26 18:50:58,973][01171] Stopping RolloutWorker_w6...
489
+ [2025-02-26 18:50:58,973][01171] Loop rollout_proc6_evt_loop terminating...
490
+ [2025-02-26 18:50:58,977][01170] Stopping RolloutWorker_w5...
491
+ [2025-02-26 18:50:58,978][00031] Component RolloutWorker_w5 stopped!
492
+ [2025-02-26 18:50:58,981][01166] Stopping RolloutWorker_w1...
493
+ [2025-02-26 18:50:58,981][00031] Component RolloutWorker_w1 stopped!
494
+ [2025-02-26 18:50:58,978][01170] Loop rollout_proc5_evt_loop terminating...
495
+ [2025-02-26 18:50:58,982][01166] Loop rollout_proc1_evt_loop terminating...
496
+ [2025-02-26 18:50:59,000][00031] Component RolloutWorker_w2 stopped!
497
+ [2025-02-26 18:50:59,000][01167] Stopping RolloutWorker_w2...
498
+ [2025-02-26 18:50:59,005][01167] Loop rollout_proc2_evt_loop terminating...
499
+ [2025-02-26 18:50:59,202][01169] Stopping RolloutWorker_w4...
500
+ [2025-02-26 18:50:59,202][00031] Component RolloutWorker_w4 stopped!
501
+ [2025-02-26 18:50:59,203][01169] Loop rollout_proc4_evt_loop terminating...
502
+ [2025-02-26 18:50:59,221][00031] Component RolloutWorker_w0 stopped!
503
+ [2025-02-26 18:50:59,223][01165] Stopping RolloutWorker_w0...
504
+ [2025-02-26 18:50:59,224][01165] Loop rollout_proc0_evt_loop terminating...
505
+ [2025-02-26 18:50:59,383][01172] Stopping RolloutWorker_w7...
506
+ [2025-02-26 18:50:59,384][01168] Stopping RolloutWorker_w3...
507
+ [2025-02-26 18:50:59,383][00031] Component RolloutWorker_w7 stopped!
508
+ [2025-02-26 18:50:59,385][01168] Loop rollout_proc3_evt_loop terminating...
509
+ [2025-02-26 18:50:59,385][00031] Component RolloutWorker_w3 stopped!
510
+ [2025-02-26 18:50:59,386][00031] Waiting for process learner_proc0 to stop...
511
+ [2025-02-26 18:50:59,383][01172] Loop rollout_proc7_evt_loop terminating...
512
+ [2025-02-26 18:51:00,198][00031] Waiting for process inference_proc0-0 to join...
513
+ [2025-02-26 18:51:00,200][00031] Waiting for process rollout_proc0 to join...
514
+ [2025-02-26 18:51:00,969][00031] Waiting for process rollout_proc1 to join...
515
+ [2025-02-26 18:51:00,971][00031] Waiting for process rollout_proc2 to join...
516
+ [2025-02-26 18:51:00,972][00031] Waiting for process rollout_proc3 to join...
517
+ [2025-02-26 18:51:01,477][00031] Waiting for process rollout_proc4 to join...
518
+ [2025-02-26 18:51:01,478][00031] Waiting for process rollout_proc5 to join...
519
+ [2025-02-26 18:51:01,479][00031] Waiting for process rollout_proc6 to join...
520
+ [2025-02-26 18:51:01,480][00031] Waiting for process rollout_proc7 to join...
521
+ [2025-02-26 18:51:01,481][00031] Batcher 0 profile tree view:
522
+ batching: 19.5622, releasing_batches: 0.0286
523
+ [2025-02-26 18:51:01,482][00031] InferenceWorker_p0-w0 profile tree view:
524
+ wait_policy: 0.0000
525
+ wait_policy_total: 58.3027
526
+ update_model: 5.5282
527
+ weight_update: 0.0023
528
+ one_step: 0.0030
529
+ handle_policy_step: 340.0363
530
+ deserialize: 10.9278, stack: 1.9846, obs_to_device_normalize: 72.5292, forward: 171.3019, send_messages: 12.6319
531
+ prepare_outputs: 54.1347
532
+ to_cpu: 33.5259
533
+ [2025-02-26 18:51:01,483][00031] Learner 0 profile tree view:
534
+ misc: 0.0059, prepare_batch: 12.7594
535
+ train: 52.6156
536
+ epoch_init: 0.0057, minibatch_init: 0.0076, losses_postprocess: 0.5134, kl_divergence: 0.5844, after_optimizer: 19.9850
537
+ calculate_losses: 18.3925
538
+ losses_init: 0.0036, forward_head: 1.0291, bptt_initial: 12.5241, tail: 0.7407, advantages_returns: 0.1913, losses: 1.4690
539
+ bptt: 2.2050
540
+ bptt_forward_core: 2.1292
541
+ update: 12.6759
542
+ clip: 0.8635
543
+ [2025-02-26 18:51:01,484][00031] RolloutWorker_w0 profile tree view:
544
+ wait_for_trajectories: 0.1455, enqueue_policy_requests: 6.9095, env_step: 374.2936, overhead: 7.1374, complete_rollouts: 0.8485
545
+ save_policy_outputs: 9.0334
546
+ split_output_tensors: 3.6183
547
+ [2025-02-26 18:51:01,485][00031] RolloutWorker_w7 profile tree view:
548
+ wait_for_trajectories: 0.1504, enqueue_policy_requests: 7.0626, env_step: 372.7928, overhead: 7.2863, complete_rollouts: 0.7316
549
+ save_policy_outputs: 9.3004
550
+ split_output_tensors: 3.6883
551
+ [2025-02-26 18:51:01,486][00031] Loop Runner_EvtLoop terminating...
552
+ [2025-02-26 18:51:01,488][00031] Runner profile tree view:
553
+ main_loop: 439.5736
554
+ [2025-02-26 18:51:01,489][00031] Collected {0: 4005888}, FPS: 9113.1
555
+ [2025-02-26 18:51:01,846][00031] Loading existing experiment configuration from /kaggle/working/train_dir/default_experiment/config.json
556
+ [2025-02-26 18:51:01,847][00031] Overriding arg 'num_workers' with value 1 passed from command line
557
+ [2025-02-26 18:51:01,848][00031] Adding new argument 'no_render'=True that is not in the saved config file!
558
+ [2025-02-26 18:51:01,848][00031] Adding new argument 'save_video'=True that is not in the saved config file!
559
+ [2025-02-26 18:51:01,849][00031] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
560
+ [2025-02-26 18:51:01,851][00031] Adding new argument 'video_name'=None that is not in the saved config file!
561
+ [2025-02-26 18:51:01,852][00031] Adding new argument 'max_num_frames'=1000000000.0 that is not in the saved config file!
562
+ [2025-02-26 18:51:01,852][00031] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
563
+ [2025-02-26 18:51:01,853][00031] Adding new argument 'push_to_hub'=False that is not in the saved config file!
564
+ [2025-02-26 18:51:01,854][00031] Adding new argument 'hf_repository'=None that is not in the saved config file!
565
+ [2025-02-26 18:51:01,854][00031] Adding new argument 'policy_index'=0 that is not in the saved config file!
566
+ [2025-02-26 18:51:01,855][00031] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
567
+ [2025-02-26 18:51:01,857][00031] Adding new argument 'train_script'=None that is not in the saved config file!
568
+ [2025-02-26 18:51:01,857][00031] Adding new argument 'enjoy_script'=None that is not in the saved config file!
569
+ [2025-02-26 18:51:01,858][00031] Using frameskip 1 and render_action_repeat=4 for evaluation
570
+ [2025-02-26 18:51:01,888][00031] Doom resolution: 160x120, resize resolution: (128, 72)
571
+ [2025-02-26 18:51:01,891][00031] RunningMeanStd input shape: (3, 72, 128)
572
+ [2025-02-26 18:51:01,893][00031] RunningMeanStd input shape: (1,)
573
+ [2025-02-26 18:51:01,906][00031] ConvEncoder: input_channels=3
574
+ [2025-02-26 18:51:02,010][00031] Conv encoder output size: 512
575
+ [2025-02-26 18:51:02,011][00031] Policy head output size: 512
576
+ [2025-02-26 18:51:02,211][00031] Loading state from checkpoint /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
577
+ [2025-02-26 18:51:03,031][00031] Num frames 100...
578
+ [2025-02-26 18:51:03,151][00031] Num frames 200...
579
+ [2025-02-26 18:51:03,263][00031] Num frames 300...
580
+ [2025-02-26 18:51:03,385][00031] Num frames 400...
581
+ [2025-02-26 18:51:03,509][00031] Num frames 500...
582
+ [2025-02-26 18:51:03,597][00031] Avg episode rewards: #0: 8.270, true rewards: #0: 5.270
583
+ [2025-02-26 18:51:03,598][00031] Avg episode reward: 8.270, avg true_objective: 5.270
584
+ [2025-02-26 18:51:03,685][00031] Num frames 600...
585
+ [2025-02-26 18:51:03,806][00031] Num frames 700...
586
+ [2025-02-26 18:51:03,925][00031] Num frames 800...
587
+ [2025-02-26 18:51:04,043][00031] Num frames 900...
588
+ [2025-02-26 18:51:04,164][00031] Num frames 1000...
589
+ [2025-02-26 18:51:04,227][00031] Avg episode rewards: #0: 7.035, true rewards: #0: 5.035
590
+ [2025-02-26 18:51:04,228][00031] Avg episode reward: 7.035, avg true_objective: 5.035
591
+ [2025-02-26 18:51:04,335][00031] Num frames 1100...
592
+ [2025-02-26 18:51:04,452][00031] Num frames 1200...
593
+ [2025-02-26 18:51:04,569][00031] Num frames 1300...
594
+ [2025-02-26 18:51:04,692][00031] Num frames 1400...
595
+ [2025-02-26 18:51:04,774][00031] Avg episode rewards: #0: 6.410, true rewards: #0: 4.743
596
+ [2025-02-26 18:51:04,775][00031] Avg episode reward: 6.410, avg true_objective: 4.743
597
+ [2025-02-26 18:51:04,871][00031] Num frames 1500...
598
+ [2025-02-26 18:51:04,998][00031] Num frames 1600...
599
+ [2025-02-26 18:51:05,116][00031] Num frames 1700...
600
+ [2025-02-26 18:51:05,234][00031] Num frames 1800...
601
+ [2025-02-26 18:51:05,354][00031] Num frames 1900...
602
+ [2025-02-26 18:51:05,451][00031] Avg episode rewards: #0: 6.838, true rewards: #0: 4.837
603
+ [2025-02-26 18:51:05,452][00031] Avg episode reward: 6.838, avg true_objective: 4.837
604
+ [2025-02-26 18:51:05,529][00031] Num frames 2000...
605
+ [2025-02-26 18:51:05,652][00031] Num frames 2100...
606
+ [2025-02-26 18:51:05,778][00031] Num frames 2200...
607
+ [2025-02-26 18:51:05,899][00031] Num frames 2300...
608
+ [2025-02-26 18:51:05,977][00031] Avg episode rewards: #0: 6.238, true rewards: #0: 4.638
609
+ [2025-02-26 18:51:05,977][00031] Avg episode reward: 6.238, avg true_objective: 4.638
610
+ [2025-02-26 18:51:06,072][00031] Num frames 2400...
611
+ [2025-02-26 18:51:06,199][00031] Num frames 2500...
612
+ [2025-02-26 18:51:06,321][00031] Num frames 2600...
613
+ [2025-02-26 18:51:06,440][00031] Num frames 2700...
614
+ [2025-02-26 18:51:06,498][00031] Avg episode rewards: #0: 5.838, true rewards: #0: 4.505
615
+ [2025-02-26 18:51:06,499][00031] Avg episode reward: 5.838, avg true_objective: 4.505
616
+ [2025-02-26 18:51:06,622][00031] Num frames 2800...
617
+ [2025-02-26 18:51:06,742][00031] Num frames 2900...
618
+ [2025-02-26 18:51:06,868][00031] Num frames 3000...
619
+ [2025-02-26 18:51:07,031][00031] Avg episode rewards: #0: 5.553, true rewards: #0: 4.410
620
+ [2025-02-26 18:51:07,032][00031] Avg episode reward: 5.553, avg true_objective: 4.410
621
+ [2025-02-26 18:51:07,050][00031] Num frames 3100...
622
+ [2025-02-26 18:51:07,169][00031] Num frames 3200...
623
+ [2025-02-26 18:51:07,287][00031] Num frames 3300...
624
+ [2025-02-26 18:51:07,409][00031] Num frames 3400...
625
+ [2025-02-26 18:51:07,537][00031] Num frames 3500...
626
+ [2025-02-26 18:51:07,639][00031] Avg episode rewards: #0: 5.544, true rewards: #0: 4.419
627
+ [2025-02-26 18:51:07,640][00031] Avg episode reward: 5.544, avg true_objective: 4.419
628
+ [2025-02-26 18:51:07,717][00031] Num frames 3600...
629
+ [2025-02-26 18:51:07,838][00031] Num frames 3700...
630
+ [2025-02-26 18:51:07,965][00031] Num frames 3800...
631
+ [2025-02-26 18:51:08,092][00031] Num frames 3900...
632
+ [2025-02-26 18:51:08,219][00031] Num frames 4000...
633
+ [2025-02-26 18:51:08,346][00031] Num frames 4100...
634
+ [2025-02-26 18:51:08,496][00031] Avg episode rewards: #0: 6.194, true rewards: #0: 4.639
635
+ [2025-02-26 18:51:08,497][00031] Avg episode reward: 6.194, avg true_objective: 4.639
636
+ [2025-02-26 18:51:08,527][00031] Num frames 4200...
637
+ [2025-02-26 18:51:08,651][00031] Num frames 4300...
638
+ [2025-02-26 18:51:08,769][00031] Num frames 4400...
639
+ [2025-02-26 18:51:08,894][00031] Num frames 4500...
640
+ [2025-02-26 18:51:09,010][00031] Num frames 4600...
641
+ [2025-02-26 18:51:09,128][00031] Num frames 4700...
642
+ [2025-02-26 18:51:09,279][00031] Avg episode rewards: #0: 6.483, true rewards: #0: 4.783
643
+ [2025-02-26 18:51:09,280][00031] Avg episode reward: 6.483, avg true_objective: 4.783
644
+ [2025-02-26 18:51:22,784][00031] Replay video saved to /kaggle/working/train_dir/default_experiment/replay.mp4!
645
+ [2025-02-26 18:53:06,496][00031] Loading existing experiment configuration from /kaggle/working/train_dir/default_experiment/config.json
646
+ [2025-02-26 18:53:06,497][00031] Overriding arg 'num_workers' with value 1 passed from command line
647
+ [2025-02-26 18:53:06,498][00031] Adding new argument 'no_render'=True that is not in the saved config file!
648
+ [2025-02-26 18:53:06,499][00031] Adding new argument 'save_video'=True that is not in the saved config file!
649
+ [2025-02-26 18:53:06,499][00031] Adding new argument 'video_frames'=1000000000.0 that is not in the saved config file!
650
+ [2025-02-26 18:53:06,501][00031] Adding new argument 'video_name'=None that is not in the saved config file!
651
+ [2025-02-26 18:53:06,502][00031] Adding new argument 'max_num_frames'=100000 that is not in the saved config file!
652
+ [2025-02-26 18:53:06,503][00031] Adding new argument 'max_num_episodes'=10 that is not in the saved config file!
653
+ [2025-02-26 18:53:06,504][00031] Adding new argument 'push_to_hub'=True that is not in the saved config file!
654
+ [2025-02-26 18:53:06,505][00031] Adding new argument 'hf_repository'='francescosabbarese/rl_course_vizdoom_health_gathering_supreme' that is not in the saved config file!
655
+ [2025-02-26 18:53:06,506][00031] Adding new argument 'policy_index'=0 that is not in the saved config file!
656
+ [2025-02-26 18:53:06,507][00031] Adding new argument 'eval_deterministic'=False that is not in the saved config file!
657
+ [2025-02-26 18:53:06,508][00031] Adding new argument 'train_script'=None that is not in the saved config file!
658
+ [2025-02-26 18:53:06,509][00031] Adding new argument 'enjoy_script'=None that is not in the saved config file!
659
+ [2025-02-26 18:53:06,509][00031] Using frameskip 1 and render_action_repeat=4 for evaluation
660
+ [2025-02-26 18:53:06,533][00031] RunningMeanStd input shape: (3, 72, 128)
661
+ [2025-02-26 18:53:06,535][00031] RunningMeanStd input shape: (1,)
662
+ [2025-02-26 18:53:06,545][00031] ConvEncoder: input_channels=3
663
+ [2025-02-26 18:53:06,583][00031] Conv encoder output size: 512
664
+ [2025-02-26 18:53:06,584][00031] Policy head output size: 512
665
+ [2025-02-26 18:53:06,595][00031] Loading state from checkpoint /kaggle/working/train_dir/default_experiment/checkpoint_p0/checkpoint_000000978_4005888.pth...
666
+ [2025-02-26 18:53:07,020][00031] Num frames 100...
667
+ [2025-02-26 18:53:07,139][00031] Num frames 200...
668
+ [2025-02-26 18:53:07,259][00031] Num frames 300...
669
+ [2025-02-26 18:53:07,379][00031] Num frames 400...
670
+ [2025-02-26 18:53:07,502][00031] Num frames 500...
671
+ [2025-02-26 18:53:07,623][00031] Num frames 600...
672
+ [2025-02-26 18:53:07,728][00031] Avg episode rewards: #0: 10.400, true rewards: #0: 6.400
673
+ [2025-02-26 18:53:07,729][00031] Avg episode reward: 10.400, avg true_objective: 6.400
674
+ [2025-02-26 18:53:07,797][00031] Num frames 700...
675
+ [2025-02-26 18:53:07,914][00031] Num frames 800...
676
+ [2025-02-26 18:53:08,032][00031] Num frames 900...
677
+ [2025-02-26 18:53:08,151][00031] Num frames 1000...
678
+ [2025-02-26 18:53:08,233][00031] Avg episode rewards: #0: 7.120, true rewards: #0: 5.120
679
+ [2025-02-26 18:53:08,234][00031] Avg episode reward: 7.120, avg true_objective: 5.120
680
+ [2025-02-26 18:53:08,324][00031] Num frames 1100...
681
+ [2025-02-26 18:53:08,438][00031] Num frames 1200...
682
+ [2025-02-26 18:53:08,556][00031] Num frames 1300...
683
+ [2025-02-26 18:53:08,675][00031] Num frames 1400...
684
+ [2025-02-26 18:53:08,795][00031] Num frames 1500...
685
+ [2025-02-26 18:53:08,914][00031] Num frames 1600...
686
+ [2025-02-26 18:53:09,042][00031] Avg episode rewards: #0: 8.213, true rewards: #0: 5.547
687
+ [2025-02-26 18:53:09,043][00031] Avg episode reward: 8.213, avg true_objective: 5.547
688
+ [2025-02-26 18:53:09,086][00031] Num frames 1700...
689
+ [2025-02-26 18:53:09,209][00031] Num frames 1800...
690
+ [2025-02-26 18:53:09,326][00031] Num frames 1900...
691
+ [2025-02-26 18:53:09,444][00031] Num frames 2000...
692
+ [2025-02-26 18:53:09,558][00031] Avg episode rewards: #0: 7.120, true rewards: #0: 5.120
693
+ [2025-02-26 18:53:09,559][00031] Avg episode reward: 7.120, avg true_objective: 5.120
694
+ [2025-02-26 18:53:09,623][00031] Num frames 2100...
695
+ [2025-02-26 18:53:09,745][00031] Num frames 2200...
696
+ [2025-02-26 18:53:09,866][00031] Num frames 2300...
697
+ [2025-02-26 18:53:09,990][00031] Num frames 2400...
698
+ [2025-02-26 18:53:10,160][00031] Avg episode rewards: #0: 6.792, true rewards: #0: 4.992
699
+ [2025-02-26 18:53:10,161][00031] Avg episode reward: 6.792, avg true_objective: 4.992
700
+ [2025-02-26 18:53:10,168][00031] Num frames 2500...
701
+ [2025-02-26 18:53:10,284][00031] Num frames 2600...
702
+ [2025-02-26 18:53:10,401][00031] Num frames 2700...
703
+ [2025-02-26 18:53:10,519][00031] Num frames 2800...
704
+ [2025-02-26 18:53:10,670][00031] Avg episode rewards: #0: 6.300, true rewards: #0: 4.800
705
+ [2025-02-26 18:53:10,671][00031] Avg episode reward: 6.300, avg true_objective: 4.800
706
+ [2025-02-26 18:53:10,697][00031] Num frames 2900...
707
+ [2025-02-26 18:53:10,816][00031] Num frames 3000...
708
+ [2025-02-26 18:53:10,936][00031] Num frames 3100...
709
+ [2025-02-26 18:53:11,059][00031] Num frames 3200...
710
+ [2025-02-26 18:53:11,195][00031] Avg episode rewards: #0: 5.949, true rewards: #0: 4.663
711
+ [2025-02-26 18:53:11,196][00031] Avg episode reward: 5.949, avg true_objective: 4.663
712
+ [2025-02-26 18:53:11,242][00031] Num frames 3300...
713
+ [2025-02-26 18:53:11,366][00031] Num frames 3400...
714
+ [2025-02-26 18:53:11,491][00031] Num frames 3500...
715
+ [2025-02-26 18:53:11,617][00031] Num frames 3600...
716
+ [2025-02-26 18:53:11,737][00031] Num frames 3700...
717
+ [2025-02-26 18:53:11,841][00031] Avg episode rewards: #0: 5.930, true rewards: #0: 4.680
718
+ [2025-02-26 18:53:11,842][00031] Avg episode reward: 5.930, avg true_objective: 4.680
719
+ [2025-02-26 18:53:11,911][00031] Num frames 3800...
720
+ [2025-02-26 18:53:12,030][00031] Num frames 3900...
721
+ [2025-02-26 18:53:12,150][00031] Num frames 4000...
722
+ [2025-02-26 18:53:12,267][00031] Num frames 4100...
723
+ [2025-02-26 18:53:12,357][00031] Avg episode rewards: #0: 5.698, true rewards: #0: 4.587
724
+ [2025-02-26 18:53:12,358][00031] Avg episode reward: 5.698, avg true_objective: 4.587
725
+ [2025-02-26 18:53:12,441][00031] Num frames 4200...
726
+ [2025-02-26 18:53:12,558][00031] Num frames 4300...
727
+ [2025-02-26 18:53:12,682][00031] Num frames 4400...
728
+ [2025-02-26 18:53:12,795][00031] Avg episode rewards: #0: 5.548, true rewards: #0: 4.448
729
+ [2025-02-26 18:53:12,796][00031] Avg episode reward: 5.548, avg true_objective: 4.448
730
+ [2025-02-26 18:53:24,771][00031] Replay video saved to /kaggle/working/train_dir/default_experiment/replay.mp4!