Skip to content

add multiprocessing with asyncio, faker, uvloop, client sessions, error handling, and type hints#32

Open
farnswj1 wants to merge 3 commits intoCybrZone:mainfrom
farnswj1:main
Open

add multiprocessing with asyncio, faker, uvloop, client sessions, error handling, and type hints#32
farnswj1 wants to merge 3 commits intoCybrZone:mainfrom
farnswj1:main

Conversation

@farnswj1
Copy link

@farnswj1 farnswj1 commented Jul 6, 2024

The script has been heavily modified to include the following:

  • Multiprocessing with asyncio to take advantage of the CPU cores by having each process run their own async event loops. Each loop with have 100 tasks, each running 100 requests. For IO-bound operations, such as HTTP requests, asyncio is preferred over threading.
  • Uvloop has been installed in order to speed up performance. This is preferred over the default asyncio event loop.
  • Faker has been added to simplify generating fake data.
  • Exception handling for failed requests to ensure any response errors are handled properly.
  • Each process has its own client session, which is shared among all the tasks and requests within the event loop.
  • Type hinting and docstrings for better code documentation.

A requirements.txt was included to ensure all dependencies and their versions are documented.

@farnswj1 farnswj1 changed the title add multiprocessing with asyncio, faker, uvloop, client sessions, and type hints add multiprocessing with asyncio, faker, uvloop, client sessions, error handling, and type hints Jul 6, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant