r18.dev stopped working

[arno56]

Hi,

Anyone having issues with r18.dev scrapping?
It looked like something changed in their search pattern so codes are not matching at all.

[arno56]

The url is accessible but it doesn't parse the information.
For example with https://r18.dev/videos/vod/movies/detail/-/dvd_id=APAK-296/json

[APAK-296] [Get-R18DevUrl] Error occured on [GET] on URL | [https://r18.dev/videos/vod/movies/detail/-/dvd_id=APAK-296/json]: Just a | moment...*{box-sizing:border-box;margin:0;padding:0}html{line-height:1.15;-webkit-text-size-adjust:100%;color:#313131;font-family:system-ui,-apple-system,BlinkMacSystemFont,Segoe UI,Roboto,Helvetica Neue,Arial,Noto Sans,sans-serif,Apple Color Emoji,Segoe UI Emoji,Segoe UI Symbol,Noto Color Emoji}body{display:flex;flex-direction:column;height:100vh;min-height:100vh}.main-content{margin:8rem auto;max-width:60rem;padding-left:1.5rem}@media (width Enable JavaScript and cookies to continue(function(){window._cf_chl_opt={cvId: '3',cZone: "r18.dev",cType: 'managed',cRay: '9040c37b1d0e65d3',cH: 'LZGgmXqD7TK9aA5PsSf3Bh_fvuyzlZyCh0.GZ1TDL0Y-1737226152-1.2.1.1-zcv2LDbCE20yqMKAe0s9iRXmZCDeyljqy_XlzHv.M39LLHNxjiZ2Cu3Wuq2EA8Y9',cUPMDTk: "\/videos\/vod\/movies\/detail\/-\/dvd_id=APAK-296\/json?__cf_chl_tk=nOziOdaP0.dYAWRegSKFQUUWodPP474ef4fH8b5N9SY-1737226152-1.0.1.1-xLjI4EwpzrNWEEhz8irAZ2f0XggLpNxMbeGhUazafDQ",cFPWv: 'g',cITimeS: '1737226152',cTTimeMs: '1000',cMTimeMs: '390000',cTplC: 0,cTplV: 5,cTplB: 'cf',cK: "",fa: "\/videos\/vod\/movies\/detail\/-\/dvd_id=APAK-296\/json?__cf_chl_f_tk=nOziOdaP0.dYAWRegSKFQUUWodPP474ef4fH8b5N9SY-1737226152-1.0.1.1-xLjI4EwpzrNWEEhz8irAZ2f0XggLpNxMbeGhUazafDQ",md: "LKuY2a4b5La_le3A3.K5nPeXqHOYY2WOx_AC4qdHHFM-1737226152-1.2.1.1-q0fEZw.yoPCRZfVtTiov3LSSzfqjhfadh5ytipMxdu93fyJ0d7Sy_Df8kdRlEtWnNaVqCYI2OpHQZ.xeXW0wOjiXqh7GezrkhpNE8myP8SsTXb2KllX6DaMxqQQ.DCmc4MI42tw7EHPZC1b.E9rCHJh2ZXL_a0toompERfJpWBHzBUgcERVMXbBfZBENFwh3XFbRaqsOjH6cM7EYRZaQ_CIuonarjAy5SkhmhvN53z6327W8lU1.mOSEMwgq6UfK0ySrAtnur4nSBTCZy2dfgfn1b4T8rNntHsyE7RdcUdTFvLgp5LUKLbELy3rx.NUUz3RMTHmDEZE9BfHgEzvhULzdNNGdLpZtgpa6GHjdyiSiX.hL1NaG8BVOmv_XPt2WA36xrfL9vALCOdojZ18W2Vu1cxRpS_HqIa3HQv1irx6mLZkKL0_92hFErhy98nLQPDE2iP9MQLbG6YlJ4WokzNklcIv82otfZHRAozFVPNVz88WxWODL2QK2.FoHywD.rq60uEVJS2bHoxmxE.j0I2FfwUgrgKLesx_BXn3Ii.V1ilKGeKqkewsJRjKk_D7y32av_ec9l3PiGyvZuVrcqvtY_D0CFlZtvO0fzIsUm66MzghvPo.sw9Z3bqFTMYdPrt8AulwflHs77oivcm85jthFnbYXx4LUu7Dj7XmY1Ed7K6yLaA28V76EENqrTiou8YjM4BJ0EHna3M277hVky6lZUBQFh0I1pFa6LPBE8oKhXXYTE0ikzMrDgkVmwBLdvVrstvU3GE5Ao34De_uX82LFCOaIkwq0GAE.5gQfSi9MbK4KxEtrV2ApTwaCdXTxV8Oxyp6c.KzW35iueRqFZB5fsk5R59_xILGE4nGAOAjKlyPbHiVEtTenUtywkSqVLb5iaMwDEVJim3qU.LphYhN5ewYGmzBCwYSstjOg3AaMSg13xbU6TwxtXRxfQm_9sDUn18Z9t9zjO87WPFtjyqb999HRWQ1uqgXZOPK2iNrRQkJTBGRQhu27mDYwjAxCLvy33lMsFBvnSVy5rl_HNyh9h.QryBx6EVctJzqHv4X5K0eIAcxuz_8IP1GJPRn9Pcs.wqDna9xzwyeCYmD7dyUSMHQqpIrD.RHzaXBokpflSKG1Z7cDcFxsjRz0.Y_pzBkF7g2IA6BWfD1VRzg87uTfdNML7z7HU2.LYlX0.NYI4p_YFfQ2Tk6K39momR1CSTSFNPFwhJyay4TXzshTBc8XgWoASkqNawx8IxFQX5DSI4o50.moR6oTFKZYkp9E4TWSOYoVXHIIHNludZGu24TRo4QZFZL5ocOE3bkDcCaxxvwPZq7HpLHFPD9Xew2Q1vASiIaBDt4m.u6KxItz8av9MwY4oufHTMrDXSid25k.c2KfvIZPGIn9Bl4dEilBmeLzoUOzgynLFkYBFb1ZRJPeEyA0h0lsVJ4ahTkUPvmMYR8wcYhPxx39BaYnWnMu3BboPq907z445m1KyC4FfoyqS827Nh_nOQIY8VZhOECiLdAygck_RnGzRoyhfyzod0uDki9OGaHr3mGU5fkydoxH7M1eAYsn2sh.P6eAprxspvhdfe1riix4e9XpTQ3HRQG36jZToXm2M2yhhID2N6lxrJo1vQ3PBC1oE3xfOlgsq1AuDyELhHfX61F9zUYwiSyZYgvptYLOJpouVwE4462dch8mGgrjXuP89Ld6m8l6PjPo3hi5r.j8JL9sUbCV7MEcQG9LG1q8.C.UW152Rg9.4N0p_lE3U9.Bi8XYYL7SB3UiyFMDTUrCOlx940gcsbVkTzkzi1aFxGCx6fNVv3XgXTzFEkpyoh9GUf0FMGQgYnDCOSzrRSsyQPhwZNcGLSUaeLLvsqU3w15U1vau.k7CrdNseP.EFqAQXnJxYrTe3Ua4uHSDS3zgOLWpce81IKJKX2DWmIHmv_wzI7g.Y3sMYfujNW1fPwVEUwZys7vhtIJnBzTMpXoICCbuTRwErPan.f_mbwAsfn6ywv4YIy0XABJEdyuTFQCcsYctgr2mG6x.fEUJVD2.tELEmGuf9cU4XbzZlBtGg437zMAzQTtcNwJQ96wwla6M4MFkAR.XgSesDcvqCTAvokgL1O.kT9C1g59_ivbwhJ8cKPkKbl4KAvwq8UWhyHqbd74eTleOKWN4BNBpwG3htf3BEVrI89W_IuiYbFnLWiRjGlPhQmzhRIWvCjDN4SfHv05n9.OI0Uygp9cAPVm7epUe7E.zgrZVU6xyk387dcGZsKe1IR9tGHSSHQq5fumvAtZiWUnBZfsi_bZZag.KUV8kvv8qBPW7vzN_jWqthLUOvU3hpR7_YQftMMlf9OYFWIMD7Xp67GIb6G183gAOcr7zSonL29ZqPG5SpuW52bJ2fXqM4A",mdrd: "mMbR6729DbnXHiCNnL1skynE5inVnBEi9LD3XRVt6x0-1737226152-1.2.1.1-LXJ4zuiOz31hpwV4YC4hdQNQELETnjQ.c7nhGExPOr.s1V3bMPzne5osd8rr9W5nddGLf6AhIISW7E3.OfRL25tQY7f36Aje1QXUGo19wWixQuq3tm_2dhQPjRBBxFMbcIQWnFf3iQ_zgkoBPmP32t5iXIRZL1qEDouP5be.pC8UsTs5c6WZF6UkLC.a7b8E_Vk2nOssmnc_2QOGUC_xwzaEFiI41VTZTVeADcSHDhwjcdY_kHkfZr0B.qSRgd_YbpEgnrjwUW0w3YUiMRgb1w8Ud4pcwiLNLz0g2OGKcPpW6WtJ3YBbNk8AHyDahN2Hbt3zMkzp71Dnx04ZEhc9XioRSbACEDQgHend_sCUz1Bf0Ec2wyTKfopstctMNFbkTvNmtzEarses_ZnMMWIjhc50SU4eqUhGnQkoWMXI6DUYlnHiqZ0NZYYeLdourMC6ukNCDqkNdFrqViAGT8Pjtkj7nsCNaL_CXDToAAkuP_2dHU3VD3UAAwq0fvYc87L0jv0Tyq2IYfVwkubwcMttIYl9g0d8PrDbgQILd7GRohQ9RNvesHDdtoEHlXgQCcLVA2TJG32YLP4kKpXXhO6votxlYic7ZUE95cEO7mI9L3ImErxl.Q0t.3gufO8R2NFNdrlppuzJ2t2g.fkFLZT2QKi6lOqQ0PNj7Qa0Xn37SOyH67euDrTpKCIhWRaLvvzSOeKO0vLIuT9gKaQhC0Gd4kkz7rmb_hS4HY81Hj_0okFqvtZJQzTG6VhK_UOye6rzMroYiBABBm8KTVrvllE323GY3zUx.9LHDc9GpyWh3KkdLyidacaArg_M1idbQ9vdrKzbVG8wWlwRyeBOepqOd11D15UrrZip0aDk0fvw_08jgyujep6gfYz1PpZGvPhT70QXlrn_zGRMYhWSnNAPU_q0.vhMtTjBA5u0X5UdShA4hQ4Yz3ID3v6p1mYJWrp9mgtHjO26yvz5KG2IjF.MMsCvhbwAuArA0lXTg0z2uK8BzhtPIMZEHtYeaCPIebzTrgKHg93fll2SaW0Fr4Vki1XN6xhUdOIgDYlL4e_ENanKXR3RWpK1.idJjIFafcqPP_1d4DMnj4Kn2XPnCt5_Db0TBINJjE3wGX2044ngbaKV8G9s4XT9Lnc9q0sR3Uz_kp.NHWzs7PW4HC2A3RVPIo4g6EpkwABGzDSk6F7lgNWHZ5dxDttA0RG6P8XD16lTUaETANGcdO04HOkKh6ftiI_UMV290.7UNkUCRj5aSq8D8iDN7J7gy1jsJ7ILD3iJGn5PjopqReD6F2ZjQM0xti3u_DI9vYCaioRWrjCkTKrX0U9U1n6wZY0wQCK1eI1eNWeDYnUkjjPsRDFwznU7JxQ.Bhdezl1eOpZgwE__HCN8GqND40WdXZyduPLdjtdnr15p0LfvwF_5386Js6Kq8raOM7EdJweYR8dlkzm67BqAUn3ebqTdHQQAQPwJ2t9Aun11WZ2JKUZp8vdx1fUgpOeQn515mH4nV1yJNdgyJ_ZdnLv4sR165BEaQ4zyRRivB8yD2m.0epvRTKLcbLAtlDtWkzZE51u5cPL.amg21TNAytsD2fcwxOY9lVtnRo6_4OL1uFalL9aeOQhT0bM3zbq7aPLoKzLZpjAuaegRjrkO0b9FPXUFRBq6BZoUsY9CviGH3CVfYvVndSMFO6DgSKD_khEmWuLcL.S3DlA.F0kUDkduO1vsUdehHSM4dIgJahRGznfxMUILlYGyNwFiagx5I_HD.b83Au31gGQZpfSbLrqDiCxBcdOaJE93GPVgDAj_7q7i.h5mUUFRi52SZB5V.zkc5.u6NIVXSMZhxCGp2yrkIpvvJxlyHJ0UoayRtZM.PvwjWRm.LQJjIMF4b85Ud0RgowozB067BKSIBK_wSNeTeruXLPnc.5_3KkNAn7ilr3yt99.VFb7q6jr3A3NcQdzJZp4Ppj.KZTNuvS_VCJpBCqOUSKCWgb3PsQYPgG9Cd8qU6.FJqnKTTOBleN_7cll3CVBSebJKJGlFs0oy78MeKd6JjbOfff.J5SJ_RZOCAczH27_JPo5pJZSXJaRQkD3yXDdTdfB.49w1RE2d4gZU_j6xVT3i4haVPA8lKmgh4vLdfDw4YC1nZ8.GwklCFXBa.m2H7A1s2AiBdGeB.R9GOmQvXmgA.OlIb9C8nNBxRLfpVhQsFbg.XiY5KY07phnfZVPTNSQAIcx12ae_JkNw1og7_p3J7k9Bt68nk2w8gkJJPFDebdmJr89I8fpLmrXH3uCjMvYmc5VWA7hmP4VNXCrjoARZ1UplTlTKqoB27qbkiSQZ8LfjPGpWPA7YXspVItcgHPqXkqOhXxpsrY0twSiQPh2xYKzNZWmIS9xTJBEHOBDkp_JHEHW12OARxEuOeRcOMk2JRLzQDFF1iWL8BJ5ggE8X3zomiLF.ni1fOGdEr.VJXrZyv2Sq7Hmlm4Cv2U0AfQDqHba8QM0oBWAvULUlYNdpRHHemx2MvImEEwSHyAkoiI7He8daXR_2kEUdmAaesAH6MKiGldtVFqTGqaj79NgPGvladMve8QVRU_GHX095Gby7HXIKOnU64KqmYFLdqdRVFcs_Www"};var cpo = document.createElement('script');cpo.src = '/cdn-cgi/challenge-platform/h/g/orchestrate/chl_page/v1?ray=9040c37b1d0e65d3';window._cf_chl_opt.cOgUHash = location.hash === '' && location.href.indexOf('#') !== -1 ? '#' : location.hash;window._cf_chl_opt.cOgUQuery = location.search === '' && location.href.slice(0, location.href.length - window._cf_chl_opt.cOgUHash.length).indexOf('?') !== -1 ? '?' : location.search;if (window.history && window.history.replaceState) {var ogU = location.pathname + window._cf_chl_opt.cOgUQuery + window._cf_chl_opt.cOgUHash;history.replaceState(null, null, "\/videos\/vod\/movies\/detail\/-\/dvd_id=APAK-296\/json?__cf_chl_rt_tk=nOziOdaP0.dYAWRegSKFQUUWodPP474ef4fH8b5N9SY-1737226152-1.0.1.1-xLjI4EwpzrNWEEhz8irAZ2f0XggLpNxMbeGhUazafDQ" + window._cf_chl_opt.cOgUHash);cpo.onload = function() {history.replaceState(null, null, ogU);}}document.getElementsByTagName('head')[0].appendChild(cpo);}());

[RabidWolf]

Just did a scrape and working ok here.

[arno56]

I checked the error again, this is basically a Cloudflare challenge.

I set a residential proxy but I don't even know if this feature works on Javinizer.
I don't have any access data in my proxy log.

  "proxy.enabled": true,
  "proxy.host": "brd.superproxy.io:33335",
  "proxy.username": "brd-customer-hl_********-zone-residential_proxy1",
  "proxy.password": "********",

What's wrong in that?

Thank you

[RabidWolf]

Doesn't seem to work anymore.

[CosmicThing2]

Yup same for me too

[zlrss17]

I'm just adjust "throttlelimit": 3, to 1 it's help me a lot.

[jvlflame]

I've talked with the owner of r18.dev and found that they temporarily blocked PowerShell requests from accessing the site due to high traffic basically DDOSing his site (albeit not entirely from Javinizer). I'm going to introduce a couple changes in a new release that should reduce the load on their server when scraping which includes:

• Restricting throttlelimit to a max of 3 (you should probably use 1)
• Adding a default sleep time of 5 seconds between each JAV scraper thread (configurable)
• Adding a custom useragent for r18.dev requests so that the owner can identify Javinizer traffic (and notify me if needed)

I also want to emphasize to everyone to be mindful with your sorting habits. For example, spread your sessions out and scrape a couple at a time instead of thousands at a time, use a throttle limit of 1, etc. The last thing we need is for more sites to shut down or block scrapers because some people aren't being considerate when scraping.

[arno56]

I've talked with the owner of r18.dev and found that they temporarily blocked PowerShell requests from accessing the site due to high traffic basically DDOSing his site (albeit not entirely from Javinizer). I'm going to introduce a couple changes in a new release that should reduce the load on their server when scraping which includes:

• Restricting throttlelimit to a max of 3 (you should probably use 1)
• Adding a default sleep time of 5 seconds between each JAV scraper thread (configurable)
• Adding a custom useragent for r18.dev requests so that the owner can identify Javinizer traffic (and notify me if needed)

I also want to emphasize to everyone to be mindful with your sorting habits. For example, spread your sessions out and scrape a couple at a time instead of thousands at a time, use a throttle limit of 1, etc. The last thing we need is for more sites to shut down or block scrapers because some people aren't being considerate when scraping.

It works like a charm now. Thanks for the quick update.

[picito10x]

I cannot scrape R18Dev anymore on my Synology NAS as I have to run GUI and the version 2.6.0 and 2.6.1 just would not work (same as #394). I have to run on 2.5.17-1 GUI through Docker and this does not have the R18Dev fix.

Help would be greatly appreciated!