TypeError: worker_thread() takes exactly 2 arguments (5 given)

[duyunshu]

Hi @babaktr I was trying to run your code but got error: TypeError: worker_thread() takes exactly 2 arguments (5 given).

I then tried fix it by removing some parameters, but more error occurs in the training part of the code. I'm wondering what is the status of your implementation now? Myself now is modifying miyosuda's code for one-step but so far I have difficulties reach convergence. Wondering if you have an estimation of the running time and/or convergence for Pong game?

Thanks a lot!

[babaktr]

Hey, sorry for the broken code. I've been meaning to fix it, but just haven't had the time. I'll look in to it soon!

[duyunshu]

I was able to make it run yesterday, but doesn't seems learning?

I've read people discuss about how sensitive the one step method is to the parameters and its difficult to fully replicate the results from the paper. wondering if you have any opinion on that. There aren't many replication for one-step, yous was among the most organized /clear ones

[babaktr]

You know, it kind of makes me happy to hear that it isn't just me who's having problems with replicating the results of the paper. Last time I was playing around with this implementation, it was learning (after a long long time), just not nearly as good as it appears from the somewhat cryptic results from DeepMind. Did you encounter this discussion in a specific forum or so? Mind sharing the link in that case?

[duyunshu]

yeah I saw the discussion here: https://gitter.im/traai/async-deep-rl

I'm running my modified 1-step Q now with 8 threads, and the score is only -14 after 8 hours. but if you look at figure 4 in the paper, it should have already gone above 0 after 8 hours. but I mean, it's still going up slowly so I'm just gonna wait longer and see.

[duyunshu]

also people have mentioned that t_max = 5 is not going to work but instead should do t_max = 32. that's my setting now.