Shared Interface¶
Gemini and ChatGPT share the same interface — everything documented here applies to both. For platform-specific behavior, see the Gemini and ChatGPT pages.
Constructor¶
Both scrapers accept the same constructor arguments. Pass these when instantiating Gemini() or ChatGPT().
hermex.scraper_base.Scraper.__init__(chrome_version=None, download_dir=Path('.'), headless=False, typing_delay=0.025, disable_web_security=True, data_dir=None)
¶
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
chrome_version
|
Chrome major version number. Defaults to auto-detecting the installed Chrome version. |
None
|
|
download_dir
|
Directory where downloaded files (e.g. generated images) are saved. |
Path('.')
|
|
headless
|
Run the browser without a visible window. |
False
|
|
typing_delay
|
Seconds between each keystroke when typing character-by-character. |
0.025
|
|
disable_web_security
|
Pass --disable-web-security to Chrome. Needed for some scrapers (e.g. ChatGPT, Gemini) but triggers bot detection on stricter sites — set False for those. |
True
|
|
data_dir
|
Root directory where Hermex stores its data. Defaults to the platform-appropriate data directory (~/.local/share/hermex on Linux, ~/Library/Application Support/hermex on macOS). Browser profiles are stored as subdirectories within this path (e.g. data_dir/chrome_profile/). |
None
|
First-time setup¶
Run setup() once per machine before using Hermex for the first time. It builds a persistent browser profile that significantly reduces bot detection risk. If you need login-gated features (e.g. file upload on Gemini), log in during this session.
hermex.scraper_base.Scraper.setup(data_dir=None)
classmethod
¶
First-time setup required before using Hermex.
Opens a browser window so you can browse around briefly. This builds a browser profile that looks like a real user, which significantly reduces bot detection risk in subsequent automated runs. Everyone must run this at least once after installation.
If you need login-gated features (e.g. file upload), log in during this session. Hermex will reuse the saved session in all future runs — repeat setup only if your session expires.
Close the browser window when done.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_dir
|
Must match the data_dir you pass to the constructor. Defaults to the platform-appropriate data directory. Usage: Gemini.setup() |
None
|
Opening and closing the browser¶
hermex.scraper_base.Scraper.open_url(url=None, timeout=30)
¶
Open a URL in the browser and wait for the page to be ready.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
url
|
URL to navigate to. |
None
|
|
timeout
|
Maximum seconds to wait for the page to be ready before raising TimeoutException. |
30
|
hermex.scraper_base.Scraper.close()
¶
Close the browser and clean up
Sending queries¶
query() is the primary method for most use cases — it handles sending, waiting, and returning the response in one call. Use send_message() + wait_until_idle() + get_last_response() separately only when you need finer control.
hermex.scraper_base.Scraper.query(message, attachments=None, paste=False, fake_typing=True, typing_delay=None, get_markdown=False, remove_watermark=False, timeout=None)
¶
Send a message, wait for the response to complete, and return it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
message
|
str
|
Text to send. |
required |
attachments
|
list[str | Path]
|
List of file paths to attach. See
|
None
|
paste
|
bool
|
If True, paste the message instead of typing it character by character. Useful for long messages where typing is too slow. |
False
|
fake_typing
|
bool
|
When paste=True, type dummy text first to avoid bot detection, then replace it with the real message. |
True
|
typing_delay
|
float
|
Seconds between each keystroke. Overrides the instance-level default. |
None
|
get_markdown
|
bool
|
If True, return the raw markdown source instead of plain text. |
False
|
remove_watermark
|
bool
|
If True, remove the watermark from any downloaded image. |
False
|
timeout
|
float
|
Maximum seconds to wait for the response before raising TimeoutException. Defaults to 5 minutes. |
None
|
Returns:
| Type | Description |
|---|---|
AssistantMessage
|
AssistantMessage with text and image fields (either may be None, but not both). |
hermex.scraper_base.Scraper.simple_query(message, attachments=None, paste=False, fake_typing=True, typing_delay=None, get_markdown=False, remove_watermark=False, timeout=None)
classmethod
¶
Open the browser, send a message, and return the response.
Convenience method for one-shot scripts that don't need a persistent session. Opens the browser, sends the message, closes the browser, and returns the full AssistantMessage.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
message
|
str
|
Text to send. |
required |
attachments
|
list[str | Path]
|
List of file paths to attach. See
|
None
|
paste
|
bool
|
If True, paste the message instead of typing it character by character. Useful for long messages where typing is too slow. |
False
|
fake_typing
|
bool
|
When paste=True, type dummy text first to avoid bot detection, then replace it with the real message. |
True
|
typing_delay
|
float
|
Seconds between each keystroke. Overrides the instance-level default. |
None
|
get_markdown
|
bool
|
If True, return the raw markdown source instead of plain text. |
False
|
remove_watermark
|
bool
|
If True, remove the watermark from any downloaded image. |
False
|
timeout
|
float
|
Maximum seconds to wait for the response before raising TimeoutException. Defaults to 5 minutes. |
None
|
Returns:
| Type | Description |
|---|---|
AssistantMessage
|
AssistantMessage with text and image fields (either may be None, but not both). |
hermex.scraper_base.Scraper.send_message(message, attachments=None, paste=False, fake_typing=True, typing_delay=None, submit=True)
abstractmethod
¶
Input a message into the chat, optionally attaching files.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
message
|
str
|
Text to send. |
required |
attachments
|
list[str | Path]
|
List of file paths to attach. See
|
None
|
paste
|
bool
|
If True, paste the message instead of typing it character by character. Useful for long messages where typing is too slow. |
False
|
fake_typing
|
bool
|
When paste=True, type dummy text first to avoid bot detection, then replace it with the real message. |
True
|
typing_delay
|
float
|
Seconds between each keystroke. Overrides the instance-level default. |
None
|
submit
|
bool
|
Whether to press Enter after composing the message. |
True
|
Waiting and responses¶
hermex.scraper_base.Scraper.wait_until_idle(timeout=None)
¶
Block until the chatbot has finished generating its response.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
timeout
|
float
|
Maximum seconds to wait before raising TimeoutException. Defaults to 5 minutes. |
None
|
hermex.scraper_base.Scraper.get_last_response(get_markdown=False, remove_watermark=False)
abstractmethod
¶
Retrieve the last response from the chat interface.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
get_markdown
|
bool
|
If True, return the raw markdown source instead of plain text. |
False
|
remove_watermark
|
bool
|
If True, remove the watermark from any downloaded image. |
False
|
Returns:
| Type | Description |
|---|---|
AssistantMessage
|
AssistantMessage with text and image fields (either may be None, but not both). |
State inspection¶
get_state() returns the current UI state of the chatbot. Useful for debugging or building custom polling logic. For most use cases, wait_until_idle() is sufficient.
hermex.scraper_base.Scraper.get_state()
abstractmethod
¶
Return the current state of the chatbot UI.
Possible states: - State.IDLE: the interface is ready and waiting for input. - State.TYPING: the input box has content that has not been submitted yet. - State.UPLOADING: a file upload is in progress. - State.GENERATING: the model is actively generating a response.
Returns:
| Type | Description |
|---|---|
State
|
A State value representing the current UI state. |
Raises:
| Type | Description |
|---|---|
Exception
|
if the state cannot be determined (e.g. expected DOM elements are missing). Callers that need to tolerate transient failures should use wait_until_idle() instead, which has built-in error tolerance. |
Utilities¶
hermex.scraper_base.Scraper.sleep(t)
¶
Sleep for approximately t seconds, with a small random jitter to appear more human-like.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
t
|
Target sleep duration in seconds. |
required |
hermex.scraper_base.Scraper.short_wait()
¶
Wait for the default short duration (7 seconds). Use after UI interactions that need a moment to settle.
hermex.scraper_base.Scraper.long_wait()
¶
Wait for the default long duration (5 minutes). Use after sending a prompt that triggers image generation or a slow response.
hermex.scraper_base.Scraper.refresh_page()
¶
Reload the current page.
hermex.scraper_base.Scraper.get_current_url(only_base=False)
¶
Return the current browser URL.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
only_base
|
If True, strip query parameters and return only the base URL. |
False
|
Data management¶
hermex.clear_data(data_dir=_default_data_dir)
¶
Delete all data stored by Hermex (browser profiles, etc.).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data_dir
|
Root data directory to remove. Defaults to the platform-appropriate Hermex data directory. Pass a custom path if you initialized scrapers with one. |
data_dir
|