106 lines
4.3 KiB
Markdown
106 lines
4.3 KiB
Markdown
# Copilot Instructions for wohn-bot
|
|
|
|
## Project Overview
|
|
|
|
A Python-based apartment monitoring bot for Berlin's public housing portal (inberlinwohnen.de) and WG rooms (wgcompany.de). Monitors listings from 6 housing companies (HOWOGE, Gewobag, Degewo, Gesobau, Stadt und Land, WBM) plus WGcompany, and sends Telegram notifications with optional auto-application via Playwright browser automation.
|
|
|
|
## Architecture
|
|
|
|
**Modularized structure** with the following key components:
|
|
|
|
- `main.py`: Entry point for the bot.
|
|
- `handlers/`: Contains company-specific handlers for auto-apply functionality. Each handler is responsible for automating the application process for a specific housing company. Includes:
|
|
- `howoge_handler.py`
|
|
- `gewobag_handler.py`
|
|
- `degewo_handler.py`
|
|
- `gesobau_handler.py`
|
|
- `stadtundland_handler.py`
|
|
- `wbm_handler.py`
|
|
- `base_handler.py`: Provides shared functionality for all handlers.
|
|
- `application_handler.py`: Delegates application tasks to the appropriate handler based on the company.
|
|
- `telegram_bot.py`: Handles Telegram bot commands and notifications.
|
|
|
|
**Data flow**: Fetch listings → Compare with `listings.json` / `wgcompany_listings.json` → Detect new → Log to CSV → Auto-apply if autopilot enabled → Save to `applications.json` → Send Telegram notification.
|
|
|
|
## Key Patterns
|
|
|
|
### Company-specific handlers
|
|
Each housing company has a dedicated handler in the `handlers/` directory. When adding support for a new company:
|
|
1. Create a new handler file in `handlers/` (e.g., `newcompany_handler.py`).
|
|
2. Implement the handler by extending `BaseHandler` and overriding necessary methods.
|
|
3. Update `application_handler.py` to include the new handler in the `handlers` dictionary.
|
|
|
|
### Listing identification
|
|
Listings are hashed by `md5(key_fields)[:12]` to generate stable IDs:
|
|
- InBerlin: `md5(rooms+size+price+address)`
|
|
- WGcompany: `md5(link+price+size)`
|
|
|
|
### State management
|
|
- `state.json` - Runtime state (autopilot toggle)
|
|
- `listings.json` - Previously seen inberlinwohnen listings
|
|
- `wgcompany_listings.json` - Previously seen WGcompany listings
|
|
- `applications.json` - Application history with success/failure status
|
|
- `listing_times.csv` / `wgcompany_times.csv` - Time-series data for pattern analysis
|
|
|
|
## Development
|
|
|
|
### Run locally
|
|
```bash
|
|
# Install dependencies (requires Playwright)
|
|
pip install -r requirements.txt
|
|
playwright install chromium
|
|
|
|
# Set env vars and run
|
|
export TELEGRAM_BOT_TOKEN=... TELEGRAM_CHAT_ID=...
|
|
python main.py
|
|
```
|
|
|
|
### Docker (production)
|
|
```bash
|
|
cp .env.example .env # Configure credentials
|
|
docker compose up -d
|
|
docker compose logs -f
|
|
```
|
|
|
|
### Debugging
|
|
- Screenshots saved to `data/` on application failures (`*_nobtn_*.png`)
|
|
- HTML saved to `data/debug_page.html` (inberlin) and `data/wgcompany_debug.html`
|
|
- Full logs in `data/monitor.log`
|
|
|
|
## Environment Variables
|
|
|
|
Required: `TELEGRAM_BOT_TOKEN`, `TELEGRAM_CHAT_ID`
|
|
InBerlin login: `INBERLIN_EMAIL`, `INBERLIN_PASSWORD`
|
|
Form data: `FORM_ANREDE`, `FORM_VORNAME`, `FORM_NACHNAME`, `FORM_EMAIL`, `FORM_PHONE`, `FORM_STRASSE`, `FORM_HAUSNUMMER`, `FORM_PLZ`, `FORM_ORT`, `FORM_PERSONS`, `FORM_CHILDREN`, `FORM_INCOME`
|
|
WGcompany: `WGCOMPANY_ENABLED`, `WGCOMPANY_MIN_SIZE`, `WGCOMPANY_MAX_SIZE`, `WGCOMPANY_MIN_PRICE`, `WGCOMPANY_MAX_PRICE`, `WGCOMPANY_BEZIRK`
|
|
|
|
## Common Tasks
|
|
|
|
### Fix a broken company handler
|
|
Check `data/*_nobtn_*.png` screenshots and `data/debug_page.html` to see actual page structure. Update selectors in the corresponding handler file in `handlers/`.
|
|
|
|
### Add Telegram command
|
|
1. Add a case in `TelegramBot._handle_update()`.
|
|
2. Implement the corresponding `_handle_{command}_command()` method.
|
|
|
|
### Modify listing extraction
|
|
- InBerlin: Update regex patterns in `InBerlinMonitor.fetch_listings()`. Test against `data/debug_page.html`.
|
|
- WGcompany: Update parsing in `WGCompanyMonitor.fetch_listings()`. Test against `data/wgcompany_debug.html`.
|
|
|
|
## Unit Tests
|
|
|
|
### Overview
|
|
The project includes unit tests to ensure functionality and reliability. Key test files:
|
|
|
|
- `tests/test_telegram_bot.py`: Tests the Telegram bot's commands and messaging functionality.
|
|
- `tests/test_error_rate_plot.py`: Tests the error rate plot generator for autopilot applications.
|
|
|
|
### Running Tests
|
|
|
|
To run the tests, use:
|
|
|
|
```bash
|
|
pytest tests/
|
|
```
|
|
|
|
Ensure all dependencies are installed and the environment is configured correctly before running the tests.
|