Skip to content

docs: improve README quickstart #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
parker-ekwis-bot wants to merge 2 commits into
base: main
Choose a base branch
from
Open

docs: improve README quickstart #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c0afbd5
docs: update README quickstart
parker-ekwis-bot Jan 27, 2026
d910bac
docs: expand README and fix env example
parker-ekwis-bot Jan 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 7 additions & 4 deletions .env.example
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
SALESFORCE_USERNAME=username@domain.com
SALESFORCE_PASSWORD=password
SALESFORCE_SECURITY_TOKEN=
SALESFORCE_DOMAIN=login / company.my
# Salesforce credentials
SALESFORCE_USERNAME=you@example.com
SALESFORCE_PASSWORD=your_password
# Security token (required unless your IP is whitelisted)
SALESFORCE_SECURITY_TOKEN=your_security_token
# Domain: "login" (production), "test" (sandbox), or your My Domain subdomain (e.g. "acme")
SALESFORCE_DOMAIN=login
230 changes: 36 additions & 194 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,219 +1,61 @@
# Salesforce Python Template

A robust Python template designed for seamless interaction with Salesforce. This project enables bulk data operations, SOQL queries, CSV processing, interactive field mapping, and data enrichment via web scraping.
A concise template for Salesforce data workflows in Python. It includes:
- CSV upload via the SObject Collections endpoint (batched up to 200 records).
- SOQL query to CSV.
- An enrichment demo that scrapes public pages and updates records after confirmation.

## Features

- **Salesforce Authentication:** Utilises environment variables for secure access
- **Bulk Data Operations:** Supports insert, update, delete, and upsert operations
- **SOQL Query Execution:** Run queries against Salesforce and export results to CSV
- **Interactive Field Mapping:** Map CSV columns to Salesforce fields during upload
- **Data Enrichment:** Enhances Salesforce records by scraping additional company information
- **Customisable Configuration:** Easily adjust settings via a YAML configuration file
- **Comprehensive Logging and Error Handling**
- **Unit and Integration Tests:** Ensuring high code quality with pytest

## Project Setup

### 1. Clone the Repository
## Quickstart
```bash
git clone <repository-url>
cd Salesforce-Python-Template
```

### 2. Create and Activate a Virtual Environment
```bash
python3 -m venv venv
python -m venv venv
source venv/bin/activate

# Confirm Python version (3.9 or higher required)
python --version
```

### 3. Install Dependencies
```bash
pip install -r requirements.txt
```

### 4. Set Up Environment Variables
Create a copy of the environment file and update it with your Salesforce credentials:

```bash
cp .env.example .env
```
# edit .env with your Salesforce credentials

Open the `.env` file in your preferred editor and configure the following variables:
```plaintext
SALESFORCE_USERNAME=your_username
SALESFORCE_PASSWORD=your_password
SALESFORCE_SECURITY_TOKEN=your_token
SALESFORCE_DOMAIN=login # Use 'test' for sandbox environments
```

## Configuration

Review and modify the configuration file located at `config/config.yaml` to suit your requirements.

### Key Configuration Areas:
- **Salesforce API Settings:** API version, batch size, and timeout
- **Logging:** Log level, format, and file destination
- **CSV Processing:** Encoding, delimiter, and directories for input and error files
- **Data Enrichment Fields:** Specify which fields to update for different Salesforce objects

### Example Configuration:
```yaml
api:
version: '57.0'
batch_size: 200
timeout: 30

logging:
level: INFO
format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
file: 'errors/app.log'

csv:
encoding: 'utf-8'
delimiter: ','
error_directory: 'errors'
input_directory: 'input'

enrichment:
fields:
Account:
- Phone
- Website
- BillingStreet
- BillingCity
- BillingState
- BillingPostalCode
- BillingCountry
Contact:
- Phone
- Email
- MailingStreet
- MailingCity
- MailingState
- MailingPostalCode
- MailingCountry
Lead:
- Phone
- Email
- Street
- City
- State
- PostalCode
- Country
python scripts/query_data.py "SELECT Id, Name FROM Account" output/accounts.csv
```

## Usage

### 1. Upload Data Script (`scripts/upload_data.py`)

This script uploads CSV data to Salesforce with interactive field mapping.
## Configuration overview
- `.env` (see `.env.example`)
- `SALESFORCE_USERNAME`, `SALESFORCE_PASSWORD`, `SALESFORCE_SECURITY_TOKEN`
- `SALESFORCE_DOMAIN` is `login` (prod), `test` (sandbox), or your My Domain subdomain
- `config/config.yaml`
- `api`: API version, default batch size, timeout
- `logging`: log level and log file (default `errors/app.log`)
- `csv`: encoding/delimiter and error/input directories
- `enrichment`: default fields per object for `scripts/manipulate_data.py`

#### Supported Operations:
- insert
- update
- delete
- upsert
## Scripts
- `scripts/upload_data.py`
- Uses SObject Collections (composite/sobjects), not the Bulk API.
- Interactive field mapping; insert/update/delete/upsert.
- Upsert requires `--external_id_field`.
- Writes per-record results to `results/` (success and error CSVs).
- `scripts/query_data.py`: runs SOQL and writes a CSV.
- `scripts/manipulate_data.py`: looks up a record, scrapes public pages, and proposes updates for confirmation.

#### Features:
- **Field Mapping:** Interactive CSV headers to Salesforce fields mapping
- **Batch Processing:** Records processed in configurable batches (up to 200 records per call)
- **Flexible Operations:** Support for all major bulk operations

#### Command-Line Options:
| Option | Description | Required |
|--------|-------------|----------|
| `csv_file` | Path to the CSV file | Yes |
| `object_name` | Salesforce object name (e.g., Account, Contact) | Yes |
| `--operation` | Operation type (insert, update, delete, upsert) | No (default: insert) |
| `--config` | Path to configuration file | No (default: config/config.yaml) |
| `--external_id_field` | External ID field for upsert | Only for upsert |
| `--batch_size` | Number of records per API call | No |

#### Example Usage:
### Usage examples
```bash
source venv/bin/activate
python scripts/upload_data.py path/to/input.csv Account --operation insert
```

### 2. Query Data Script (`scripts/query_data.py`)

Execute SOQL queries and export results to CSV.

#### Command-Line Options:
| Option | Description | Required |
|--------|-------------|----------|
| `soql` | SOQL query (in quotes) | Yes |
| `output_file` | CSV output file path | Yes |
| `--config` | Path to configuration file | No |

#### Example Usage:
```bash
source venv/bin/activate
python scripts/upload_data.py path/to/input.csv Case --operation upsert --external_id_field CaseNumber
python scripts/query_data.py "SELECT Id, Name FROM Account" output/accounts.csv
python scripts/manipulate_data.py 0016g000009yzfpAAA Account --fields Phone Website
```

### 3. Data Enrichment Script (`scripts/manipulate_data.py`)

Enrich Salesforce records with additional data through web scraping.

#### Command-Line Options:
| Option | Description | Required |
|--------|-------------|----------|
| `record_id` | Salesforce Record ID | Yes |
| `sobject_type` | Object type (Account, Contact, Lead) | Yes |
| `--config` | Path to configuration file | No |
| `--fields` | Specific fields to update | No |

#### Example Usage:
```bash
source venv/bin/activate
python scripts/manipulate_data.py 0016g000009yzfpAAA Account --fields Phone Website BillingStreet
```
## Salesforce client helpers
`src/salesforce.py` includes Bulk API helpers (`bulk_insert`, `bulk_update`) and a `query` helper. The upload script does not use these bulk helpers; it batches records through the SObject Collections REST endpoint.

## Testing
- Unit tests: `tests/test_salesforce.py`, `tests/test_bulkification.py`, `tests/test_manipulate_data.py` (with mocked HTTP).
- Integration tests: `tests/test_upload_operations.py` runs live against Salesforce and creates/deletes Case records. `tests/test_manipulate_data.py::test_full_integration` is marked integration but skipped by default.

Run the complete test suite:
Run all tests (requires Salesforce credentials and will create/delete Case records):
```bash
pytest tests/
```

Generate test coverage report:
```bash
pytest --cov=src tests/
pytest
```

## Troubleshooting

### Common Issues and Solutions:

1. **ModuleNotFoundError for 'src'**
- Run scripts from project root
- Adjust PYTHONPATH if needed

2. **Authentication Failures**
- Verify Salesforce credentials in `.env`
- Confirm API access is enabled

3. **API Errors**
- Check API version in `config/config.yaml`
- Verify user permissions in Salesforce

## Contributing

We welcome contributions! Please follow these steps:

1. Fork the repository
2. Create a feature branch
3. Submit a pull request with detailed descriptions

## License

This project is licensed under the MIT License.

---

Thank you for using the Salesforce Python Template. We hope this tool streamlines your Salesforce data operations and enrichments. For any issues or further assistance, please open an issue on GitHub.
- `ModuleNotFoundError: src`: run scripts from the project root.
- Authentication errors: confirm `.env` values, domain (`login`/`test`/My Domain), and API access.