first commit
Some checks failed
ci / deploy (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (javascript-typescript) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled

This commit is contained in:
Vlastislav Svatek
2026-06-05 10:39:05 +02:00
commit 673e67106e
217 changed files with 76612 additions and 0 deletions

15
.github/FUNDING.yml vendored Normal file
View File

@@ -0,0 +1,15 @@
# These are supported funding model platforms
github: # Replace with up to 4 GitHub Sponsors-enabled usernames e.g., [user1, user2]
patreon: # Replace with a single Patreon username
open_collective: # Replace with a single Open Collective username
ko_fi: # Replace with a single Ko-fi username
tidelift: # Replace with a single Tidelift platform-name/package-name e.g., npm/babel
community_bridge: # Replace with a single Community Bridge project-name e.g., cloud-foundry
liberapay: # Replace with a single Liberapay username
issuehunt: # Replace with a single IssueHunt username
lfx_crowdfunding: # Replace with a single LFX Crowdfunding project-name e.g., cloud-foundry
polar: # Replace with a single Polar username
buy_me_a_coffee: bonzo81 # Replace with a single Buy Me a Coffee username
thanks_dev: # Replace with a single thanks.dev username
custom: # Replace with up to 4 custom sponsorship URLs e.g., ['link1', 'link2']

61
.github/ISSUE_TEMPLATE/bug_report.yaml vendored Normal file
View File

@@ -0,0 +1,61 @@
---
name: 🐛 Bug Report
description: Report a reproducible bug in the current release of NetBox Librenms Plugin
labels: ["type: bug"]
body:
- type: markdown
attributes:
value: >
**NOTE:** This form is only for reporting _reproducible bugs_ in a current NetBox Librenms Plugin release.
- type: input
attributes:
label: NetBox Librenms Plugin version
description: What version of NetBox Librenms Plugin are you currently running?
placeholder: v0.1.0
validations:
required: true
- type: input
attributes:
label: NetBox version
description: What version of NetBox are you currently running?
placeholder: v3.6.0
validations:
required: true
- type: dropdown
attributes:
label: Python version
description: What version of Python are you currently running?
options:
- "3.8"
- "3.9"
- "3.10"
- "3.11"
- "3.12"
- "3.13"
validations:
required: true
- type: textarea
attributes:
label: Steps to Reproduce
description: >
Please provide a minimal working example to demonstrate the bug. Ensure that your example is as concise as possible
while adequately illustrating the issue.
_Please refrain from including any confidential or sensitive
information in your example._
validations:
required: true
- type: textarea
attributes:
label: Expected Behavior
description: What did you expect to happen?
placeholder: The script should execute without raising any errors or exceptions
validations:
required: true
- type: textarea
attributes:
label: Observed Behavior
description: What happened instead?
placeholder: A TypeError exception was raised
validations:
required: true

12
.github/ISSUE_TEMPLATE/config.yml vendored Normal file
View File

@@ -0,0 +1,12 @@
# Reference: https://help.github.com/en/github/building-a-strong-community/configuring-issue-templates-for-your-repository#configuring-the-template-chooser
blank_issues_enabled: false
contact_links:
- name: 📖 Contributing Policy
url: https://github.com/bonzo81/netbox-librenms-plugin/blob/master/docs/contributing.md
about: "Please read through our contributing policy before opening an issue or pull request."
- name: ❓ Discussion
url: https://github.com/bonzo81/netbox-librenms-plugin/discussions
about: "If you're just looking for help, try starting a discussion instead."
- name: 💬 Community Slack
url: https://netdev.chat
about: "Join #netbox on the NetDev Community Slack for assistance with installation issues and other problems."

View File

@@ -0,0 +1,59 @@
---
name: ✨ Feature Request
description: Propose a new NetBox Librenms Plugin feature or enhancement
labels: ["type: feature"]
body:
- type: markdown
attributes:
value: >
**NOTE:** This form is only for submitting well-formed proposals to extend or modify
NetBox Librenms Plugin in some way. If you're trying to solve a problem but can't figure out how, or if
you still need time to work on the details of a proposed new feature, please start a
[discussion](https://github.com/bonzo81/netbox-librenms-plugin/discussions) instead.
- type: input
attributes:
label: NetBox Librenms Plugin version
description: What version of NetBox Librenms Plugin are you currently running?
placeholder: v0.1.0
validations:
required: true
- type: input
attributes:
label: NetBox version
description: What version of NetBox are you currently running?
placeholder: v3.6.0
validations:
required: true
- type: dropdown
attributes:
label: Feature type
options:
- Data model extension
- New functionality
- Change to existing functionality
validations:
required: true
- type: textarea
attributes:
label: Proposed functionality
description: >
Describe in detail the new feature or behavior you are proposing. Include any specific changes
to work flows, data models, and/or the user interface. The more detail you provide here, the
greater chance your proposal has of being discussed. Feature requests which don't include an
actionable implementation plan will be rejected.
validations:
required: true
- type: textarea
attributes:
label: Use case
description: >
Explain how adding this functionality would benefit NetBox Librenms Plugin users. What need does it address?
validations:
required: true
- type: textarea
attributes:
label: External dependencies
description: >
List any new dependencies on external libraries or services that this new feature would
introduce. For example, does the proposal require the installation of a new Python package?
(Not all new features introduce new dependencies.)

View File

@@ -0,0 +1,24 @@
---
name: 🏡 Housekeeping
description: A change pertaining to the codebase itself (developers only)
labels: ["type: housekeeping"]
body:
- type: markdown
attributes:
value: >
**NOTE:** This template is for use by maintainers only. Please do not submit
an issue using this template unless you have been specifically asked to do so.
- type: textarea
attributes:
label: Proposed Changes
description: >
Describe in detail the new feature or behavior you'd like to propose.
Include any specific changes to work flows, data models, or the user interface.
validations:
required: true
- type: textarea
attributes:
label: Justification
description: Please provide justification for the proposed change(s).
validations:
required: true

75
.github/copilot-instructions.md vendored Normal file
View File

@@ -0,0 +1,75 @@
# NetBox LibreNMS Plugin AI Assistant Guide
> **Note:** Additional context-specific instructions are in `.github/instructions/`:
> - [testing.instructions.md](instructions/testing.instructions.md) applies to `tests/**`
> - [frontend.instructions.md](instructions/frontend.instructions.md) applies to templates and static files
> - [background-jobs.instructions.md](instructions/background-jobs.instructions.md) applies to `jobs.py`, import views, and import utilities
> - [sync.instructions.md](instructions/sync.instructions.md) applies to sync views, base views, tables, and sync JS
## Architecture & Key Modules
- Plugin hooks into NetBox (Django 5) under `netbox_librenms_plugin/`; respect NetBox plugin APIs (`navigation.py`, `urls.py`, `api/`).
- LibreNMS communication lives in `librenms_api.py`; reuse this client instead of new `requests` calls. It handles multi-server configs via `LibreNMSSettings` model and the `servers` plugin config, plus caching via Django cache + custom fields.
- Views follow a three-layer structure:
- **Base views** (`views/base/`) — abstract views for each sync resource (`BaseInterfaceTableView`, `BaseCableTableView`, `BaseIPAddressTableView`, `BaseVLANTableView`).
- **Object sync views** (`views/object_sync/`) — concrete per-model views registered as tabs on NetBox's Device/VM detail pages via `@register_model_view(Device, ...)`. These wire base views to models.
- **Sync action views** (`views/sync/`) — POST-only views that apply changes (add/change/delete NetBox objects). Includes `interfaces.py`, `cables.py`, `ip_addresses.py`, `vlans.py`, `devices.py`, `device_fields.py`, `locations.py`.
- **Shared mixins** (`views/mixins.py`) — `LibreNMSPermissionMixin`, `NetBoxObjectPermissionMixin`, `LibreNMSAPIMixin`, `CacheMixin`, `VlanAssignmentMixin`.
- All four sync resources (interfaces, cables, IP addresses, VLANs) follow the same three-layer pattern. VLAN sync additionally uses `VlanAssignmentMixin` for VLAN group scope resolution (Rack → Location → Site → SiteGroup → Region → Global).
- New views should extend the closest base class and compose mixins.
- Tables (`tables/*.py`) and templates (`templates/netbox_librenms_plugin/`) drive the UI. See `frontend.instructions.md` for HTMX, template, and styling conventions.
- Forms (`forms.py`) include dynamic LibreNMS API-populated choices (location dropdowns, poller groups) and a split-form pattern for settings (server config form + import settings form).
- `import_validation_helpers.py` centralizes validation state mutation during import (role/cluster/rack assignment, issue removal, status recalculation).
## Data & Sync Conventions
- Devices/VMs map to LibreNMS via the `librenms_id` custom field, then cached if absent. Always call `LibreNMSAPI.get_librenms_id` instead of touching the field directly.
- Matching is intentionally **exact-only** for site, platform, device type, and role. See `utils.py` (`find_matching_site`, `match_librenms_hardware_to_device_type`, `find_matching_platform`). Do not add fuzzy matching.
- Sync pipelines generally fetch LibreNMS data (`librenms_api.py`), cache it (`CacheMixin`), build comparison tables (`tables/`), and render HTMX fragments (`templates/netbox_librenms_plugin/htmx/`). Follow that flow for new resources.
- Virtual chassis support uses `get_virtual_chassis_member()` for port-to-member mapping and `get_librenms_sync_device()` for VC priority-based device selection.
## Developer Workflow
- Prefer the devcontainer commands (`netbox-run`, `netbox-run-bg`, `netbox-reload`, `netbox-logs`) described in `.devcontainer/README.md`. They manage NetBox + plugin reloading.
- Static assets belong in `static/netbox_librenms_plugin/`; run NetBox's `collectstatic` when bundling, but the devcontainer handles this automatically.
## Integration Touchpoints
- REST endpoints for imports live in `views/imports/actions.py` (with the list view in `views/imports/list.py`) and surface via `urls.py`. They also emit HTMX fragments (`templates/netbox_librenms_plugin/htmx/device_import_row.html`, etc.). Keep server responses and HTMX targets in sync.
- API serializers (`api/serializers.py`) mirror models for external consumption. Update serializers and `api/views.py` together to avoid contract drift.
- Navigation and menu items are registered in `navigation.py`; extend there for new sections so NetBox renders links correctly.
## Permission System
- Uses two-tier permissions via `LibreNMSSettings` model: `view_librenmssettings` (read) and `change_librenmssettings` (write). See `docs/development/permissions.md`.
- Permission constants in `constants.py`: `PERM_VIEW_PLUGIN` and `PERM_CHANGE_PLUGIN`.
### Plugin-Level Permissions
- All views inherit `LibreNMSPermissionMixin` from `views/mixins.py`, which sets `permission_required = PERM_VIEW_PLUGIN` and provides:
- `has_write_permission()` — checks `PERM_CHANGE_PLUGIN`.
- `require_write_permission()` — returns error response (HTMX `HX-Redirect` or standard redirect) if denied.
- `require_write_permission_json()` — returns `JsonResponse(403)` if denied (for AJAX endpoints).
### Object-Level Permissions
- `NetBoxObjectPermissionMixin` adds a **second layer** of permission checking for NetBox model operations (add/change/delete on Device, Interface, VLAN, etc.).
- Views declare `required_object_permissions` dict mapping HTTP methods to `[(action, Model)]` tuples, e.g.:
```python
required_object_permissions = {"POST": [("add", VLAN), ("change", VLAN)]}
```
- Some views set `required_object_permissions` dynamically per-request (e.g., `SyncInterfacesView` switches between `Interface` and `VMInterface` based on object type).
- Provides:
- `check_object_permissions(method)` → `(bool, missing_perms_list)`
- `require_object_permissions(method)` — redirect/HTMX on failure.
- `require_object_permissions_json(method)` — JSON 403 on failure.
- `require_all_permissions(method)` — combined plugin write + object perms check (redirect/HTMX).
- `require_all_permissions_json(method)` — combined check, JSON variant.
- **Sync POST handlers** must call `require_all_permissions("POST")` (not just `require_write_permission()`) and return early if it returns a response. AJAX/JSON endpoints use `require_all_permissions_json("POST")`.
- `_get_safe_redirect_url(request)` validates referrer URLs to prevent open-redirect attacks.
### Permission Helpers for Background Jobs
- Background jobs run outside view context and cannot use view mixins. Use standalone helpers from `import_utils.py` (`check_user_permissions`, `require_permissions`). See `background-jobs.instructions.md` for details.
### API & Navigation Permissions
- API endpoints use `LibreNMSPluginPermission` class in `api/views.py` (GET=view, others=change).
- Navigation menu (`navigation.py`) has 3 groups: **Settings** (Plugin Settings, Interface Mappings), **Import** (LibreNMS Import), **Status Check** (Site & Location Sync, Device Status, VM Status). All items use `permissions=[PERM_VIEW_PLUGIN]`.
- **Background job polling requires superuser** — non-superusers fall back to synchronous mode. See `background-jobs.instructions.md` for details.
## When in Doubt
- Check docs in `docs/development/` for structure, view inheritance, mixins, and template conventions before introducing new patterns.
- Review the existing sync views (e.g., `views/sync/interfaces.py`) as reference implementations for data flow and caching patterns.
- Coordinate any schema changes through Django migrations in `migrations/` and update `models.py` + admin/pydantic representations accordingly.

14
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,14 @@
version: 2
updates:
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "weekly"
groups:
github-actions:
patterns:
- "*"
- package-ecosystem: "uv"
directory: "/"
schedule:
interval: "weekly"

View File

@@ -0,0 +1,84 @@
---
applyTo: "**/jobs.py,**/views/imports/**,**/import_utils.py,**/import_validation_helpers.py"
description: Background job architecture, import workflow, and task management patterns
---
# Background Jobs & Import Workflow
## Job Architecture
- Background jobs use NetBox's `JobRunner` base class (`netbox.jobs.JobRunner`) for long-running operations like device filtering with VC detection.
- Jobs run via Redis Queue (RQ) in Redis, separate from the database Job model. Real-time status must be checked via RQ, not the database.
## Critical Job Architecture Points
- Job UUID (`job.job_id`) is used for RQ API endpoints: `/api/core/background-tasks/{uuid}/`
- Job PK (`job.pk`) is used for database endpoints and result loading
- RQ status values: `queued`, `started`, `finished`, `stopped`, `failed` (NOT `completed`)
- Database Job status values: `pending`, `scheduled`, `running`, `completed`, `failed`, `errored` (NO `cancelled` status exists)
- Check `rq_job.is_stopped` or `rq_job.is_failed` flags in Redis for cancellation detection, not database status
## Job Cancellation Flow
1. Call `/api/core/background-tasks/{uuid}/stop/` to stop RQ job
2. Call plugin's sync endpoint `/api/plugins/librenms_plugin/jobs/{pk}/sync-status/` to update database
3. Frontend polling detects status changes and redirects appropriately
## Polling Implementation
- Poll `/api/core/background-tasks/{uuid}/` for real-time RQ status
- Update modal messages based on status: "Job queued...", "Processing...", "Job completed!"
- Handle all RQ status values explicitly to avoid infinite polling
- Use `cancelInProgress` flag to prevent polling interference during cancellation
## Superuser Requirement for Background Jobs
- NetBox's `/api/core/background-tasks/` endpoint requires **superuser** (`IsSuperuser` in `BaseRQViewSet`).
- Non-superuser users cannot poll job status; they get 403 Forbidden.
- The plugin automatically falls back to synchronous mode for non-superusers—see `should_use_background_job()` in `list.py` and `actions.py`.
- This is a NetBox core design decision, not a plugin limitation. No amount of permissions (including `core.view_job`) bypasses it.
## Import Jobs
- **`FilterDevicesJob`** — background device filtering with VC detection. `job.data` keys: `device_ids`, `total_processed`, `filters`, `server_key`, `vc_detection_enabled`, `cache_timeout`, `cached_at`, `completed`. Devices are cached individually via shared cache keys from `get_validated_device_cache_key()`.
- **`ImportDevicesJob`** — background device/VM import. Calls `bulk_import_devices_shared()` for devices and `bulk_import_vms()` for VMs. `job.data` keys: `imported_device_pks`, `imported_vm_pks`, `imported_libre_device_ids`, `imported_libre_vm_ids`, `server_key`, `total`, `success_count`, `failed_count`, `skipped_count`, `virtual_chassis_created`, `errors`, `completed`.
## Shared Cache Key Pattern
- Both synchronous and background modes use `get_validated_device_cache_key()` from `import_utils.py` to generate cache keys. This ensures `_load_job_results()` in the list view can retrieve devices regardless of which mode produced them.
- `get_active_cached_searches()` manages multi-search cache to let users run and switch between searches.
- Never hardcode cache key formats; always use the helper functions.
## Permission Checks in Jobs
- Background jobs run outside view context, so they cannot use view mixins.
- Use standalone helpers from `import_utils.py` for permission checks inside job code:
- `check_user_permissions(user, permissions)``(bool, missing_list)`
- `require_permissions(user, permissions, action_description)` — raises `PermissionDenied`.
## Custom Sync Endpoint
`api/views.py::sync_job_status()` syncs database Job status with RQ job status, needed because NetBox worker doesn't always update DB when jobs stop before processing starts.
## Import Page Flow
The import page (`LibreNMSImportView` in `views/imports/list.py`) supports two modes:
1. **Synchronous** — calls `process_device_filters()` directly, renders results inline.
2. **Background** — enqueues `FilterDevicesJob`, returns `JsonResponse` with `job_id`/`job_pk`/`poll_url`. Frontend polls and redirects to `?job_id={pk}` on completion.
Result loading: `_load_job_results(job_id)` reads `job.data["device_ids"]`, reconstructs devices from per-device cache using `get_validated_device_cache_key()`.
Filter fields: `librenms_location`, `librenms_type`, `librenms_os`, `librenms_hostname`, `librenms_sysname`, `librenms_hardware`, `enable_vc_detection`, `show_disabled`, `exclude_existing`.
## Import Action Views (`views/imports/actions.py`)
- **`DeviceImportHelperMixin`** — provides `get_validated_device_with_selections()` and `render_device_row()` for HTMX row rendering. Shared by update views.
- **`BulkImportConfirmView`** (POST) — renders confirmation modal with selected device list. Returns `htmx/bulk_import_confirm.html`.
- **`BulkImportDevicesView`** (POST) — executes import. Background mode enqueues `ImportDevicesJob`; sync mode calls `bulk_import_devices()` + `bulk_import_vms()` and returns OOB row swaps with `HX-Trigger: closeModal`.
- **`DeviceValidationDetailsView`** (GET) — renders expandable validation details via `htmx/device_validation_details.html`.
- **`DeviceVCDetailsView`** (GET) — renders VC member details via `htmx/device_vc_details.html`.
- **`DeviceRoleUpdateView`**, **`DeviceClusterUpdateView`**, **`DeviceRackUpdateView`** (POST) — per-device dropdown updates. Apply selection to validation state and return re-rendered row via `render_device_row()`.
## Key Import Utilities (`import_utils.py`)
- `process_device_filters(filters, ...)` — fetches and validates devices from LibreNMS, returns list.
- `validate_device_for_import(device, ...)` — core validation function, produces validation state dict.
- `bulk_import_devices_shared(devices, user, ...)` — shared implementation between sync and background import.
- `bulk_import_vms(vm_imports, user, ...)` — VM import implementation.
- `fetch_device_with_cache(device_id, ...)` — retrieves/caches individual device data.
- Cache key functions: `get_validated_device_cache_key()`, `get_cache_metadata_key()`, `get_active_cached_searches()`, `get_import_device_cache_key()`.
## Validation Helpers (`import_validation_helpers.py`)
Centralizes validation state mutation used by the role/cluster/rack update views:
- `apply_role_to_validation()`, `apply_cluster_to_validation()`, `apply_rack_to_validation()` — update validation state when user selects a role/cluster/rack.
- `remove_validation_issue()`, `recalculate_validation_status()` — maintain issue list and overall status.
- `fetch_model_by_id()`, `extract_device_selections()` — helpers for reading form data.

View File

@@ -0,0 +1,66 @@
---
applyTo: "netbox_librenms_plugin/templates/**,netbox_librenms_plugin/static/**"
description: Frontend patterns for templates, HTMX, and static assets
---
# Frontend Patterns
## HTMX Conventions
- HTMX 2.x is the primary async layer. Table row updates return `<tr hx-swap-oob="true">`.
- Avoid `outerHTML` swaps; use OOB or targeted `innerHTML` swaps to keep table layout intact.
- All HTMX requests and `fetch()` calls must include a CSRF token. The standard pattern is `document.querySelector('[name=csrfmiddlewaretoken]').value` (from a hidden form input). The import JS also uses `getCookie('csrftoken')` as a fallback — prefer the hidden input approach for consistency.
## Modal Implementation
- Modals use Tabler (Bootstrap-like) but **without** `bootstrap.Modal` helpers.
- Buttons target the `htmx-modal-content` element and JavaScript in `librenms_import.html` toggles the wrapper.
- Do not reintroduce `data-bs-toggle` or duplicate modal IDs.
- The import page uses `ModalManager` class and `filterModalManager` instance—always use this reference in fetch callbacks, not undefined `modalInstance` variables.
## JavaScript Fetch Patterns
- Always check `response.ok` before processing fetch responses to catch HTTP errors.
- In catch blocks, show `error.message` for debugging rather than generic messages.
- The import filter form uses fetch with `Accept: application/json, text/html`—JSON for background jobs, HTML for synchronous mode.
## Form Controls
- Device import dropdowns rely on TomSelect decorators set up elsewhere.
- Keep `<select class="device-role-select">` markup stable to preserve JS hook-up.
## Styling
- Styling assumes Tabler defaults.
- Removing `table-responsive` wrappers was deliberate to prevent dropdown clipping—do not re-add them.
## Template Structure
- Templates live in `templates/netbox_librenms_plugin/`; reuse/includes under `inc/`.
- Sync pages extend `librenms_sync_base.html`.
- Tables emit HTMX-enabled columns and buttons (`tables/*.py`), so prefer updating the table renderer in Python rather than templates when changing row actions.
## Sync Tab Template Pattern
- Each sync resource has two templates following a naming convention:
- `_<resource>_sync.html` — the tab wrapper, loaded once when the tab is selected.
- `_<resource>_sync_content.html` — the HTMX-swappable inner fragment, refreshed on data changes without a full page reload.
- Current resources: `_interface_sync`, `_cable_sync`, `_ipaddress_sync`, `_vlan_sync`.
- When adding a new sync resource, create both the wrapper and content templates following this pattern.
## HTMX Fragments
- HTMX fragments live in `templates/netbox_librenms_plugin/htmx/` and include:
- `device_import_row.html` — individual import row updates.
- `device_validation_details.html` — expandable validation details.
- `device_vc_details.html` — virtual chassis member details.
- `bulk_import_confirm.html` — import confirmation modal content.
- Keep server responses and HTMX targets in sync when modifying these fragments.
## Settings Page
- `settings.html` uses a split-form pattern: two separate Django forms (`ServerConfigForm` + `ImportSettingsForm`) sharing one page, differentiated by a hidden `form_type` field (`"server_config"` or `"import_settings"`).
- The test-connection button is an HTMX POST to `TestLibreNMSConnectionView`, returning an inline alert fragment.
## Paginator
- `inc/paginator.html` is a custom paginator that preserves tab state and `interface_name_field` in pagination URLs. Used across all sync tables.
## Import Page JavaScript (`librenms_import.js`)
- Wrapped in an IIFE with `window.LibreNMSImportInitialized` guard to prevent re-initialization during HTMX swaps.
- **`ModalManager`** class wraps Bootstrap 5 modal show/hide with fallback.
- **`pollJobStatus()`** — polls `/api/core/background-tasks/{jobId}/` every 2s, updates progress messages, handles cancel button, redirects on completion.
- **`captureSelectionState()` / `restoreSelectionState()`** — preserves checkbox state across HTMX content swaps.
- **`createCacheCountdown()`** — generic countdown timer for cache expiration display.
- **`initializeFilterForm()`** — intercepts form submit, detects JSON response (background job), starts polling.
- CSRF token extracted via `getCookie('csrftoken')` (cookie-based).

View File

@@ -0,0 +1,72 @@
---
applyTo: "**/views/base/**,**/views/object_sync/**,**/views/sync/**,**/tables/**,**/librenms_sync.js"
description: Sync page architecture, base views, and sync action patterns
---
# Sync Pages
## Three-Layer View Architecture
All four sync resources (interfaces, cables, IP addresses, VLANs) follow the same pattern:
1. **Base views** (`views/base/`) — abstract classes that define the data pipeline:
- `BaseLibreNMSSyncView` — tabbed sync page, orchestrates all tabs via abstract `get_*_context()` methods.
- `BaseInterfaceTableView` — fetch ports → enrich with VLANs → cache → compare with NetBox interfaces → render table.
- `BaseCableTableView` — fetch links → match remote devices → check cable status → render table.
- `BaseIPAddressTableView` — fetch IPs → resolve interfaces → detect existing/update/new → render table.
- `BaseVLANTableView` — fetch VLANs → compare with NetBox VLANs → auto-select groups → render table.
2. **Object sync views** (`views/object_sync/`) — wire base views to NetBox models:
- Use `@register_model_view(Device, name="librenms_sync", path="librenms-sync")` to inject as a tab on Device/VM detail pages.
- Each `get_*_context()` method creates an instance of the concrete table view, copies `request`, and calls `get_context_data()`.
- VMs skip cables and VLANs (return `None`).
3. **Sync action views** (`views/sync/`) — POST-only views that create/update/delete NetBox objects:
- Follow a consistent pattern: check permissions → read selected rows from POST → load cached data → apply changes in `transaction.atomic()` → redirect to sync tab.
## Data Pipeline (Base Views)
Every base table view follows: **fetch → cache → compare → render**.
- **Fetch:** Call LibreNMS API (e.g., `get_ports()`, `get_device_ips()`, `get_device_vlans()`).
- **Cache:** Store results via `CacheMixin` keys: `librenms_{data_type}_{model_name}_{pk}`. Also store fetch timestamp at `librenms_{data_type}_last_fetched_{model_name}_{pk}`.
- **Compare:** Match LibreNMS data against NetBox objects. Each resource implements its own comparison (interface matching by name, IP matching by address/mask, VLAN matching by VID+group).
- **Render:** Build a django-tables2 table, return a partial template (`_*_sync_content.html`).
## Sync Action View Pattern
```python
class SyncSomeResourceView(LibreNMSPermissionMixin, NetBoxObjectPermissionMixin, CacheMixin, View):
required_object_permissions = {"POST": [("add", Model), ("change", Model)]}
def post(self, request, object_type, object_id):
if error := self.require_all_permissions("POST"):
return error
# 1. Resolve object (Device or VM)
# 2. Read selected items from request.POST.getlist("select")
# 3. Load cached data from cache.get(self.get_cache_key(obj, "..."))
# 4. Apply changes inside transaction.atomic()
# 5. Redirect to sync tab with ?tab=<resource>
```
## Table Conventions (`tables/*.py`)
- Tables define HTMX-enabled columns and checkboxes. Selection uses `ToggleColumn(attrs={"input": {"name": "select"}})`.
- Constructor takes contextual params (e.g., `device`, `interface_name_field`, `vlan_groups`) to customize rendering.
- Tables set `self.tab` and `self.prefix` for multi-table pagination via `get_table_paginate_count()`.
- Row attrs include `data-*` attributes for JavaScript filtering and identification.
- VLAN columns use `render_vlans()` with hidden inputs for per-row group selection and JSON data for modals.
## Key Mixins Used by Sync Views
- **`LibreNMSAPIMixin`** — lazy-creates `LibreNMSAPI` instance via `self.librenms_api` property. Also provides `get_server_info()` for template context.
- **`CacheMixin`** — generates consistent cache keys via `get_cache_key(obj, data_type)` and `get_last_fetched_key(obj, data_type)`. Also provides `get_vlan_overrides_key(obj)` for VLAN group override persistence.
- **`VlanAssignmentMixin`** — VLAN group scope resolution: Rack → Location → Site → SiteGroup → Region → Global. Used by interface and VLAN sync for auto-selecting the most-specific VLAN group and building lookup maps.
## JavaScript (`librenms_sync.js`)
- Not wrapped in an IIFE — functions are global. Master initializer `initializeScripts()` runs on `DOMContentLoaded` and `htmx:afterSwap`.
- **Key function groups:**
- Checkbox management: `initializeTableCheckboxes()`, `updateBulkActionButton()`.
- TomSelect dropdowns: `initializeVCMemberSelect()`, `initializeVRFSelects()`, `initializeVlanGroupSelects()`, `initializeVlanSyncGroupSelects()`. Uses `TOMSELECT_INIT_DELAY_MS = 100` for delayed initialization after HTMX swaps.
- Verification: `handleInterfaceChange()`, `handleCableChange()`, `handleVRFChange()` — POST to single-item verify endpoints.
- VLAN modals: `openVlanDetailModal()`, `verifyVlanInGroup()`, `verifyVlanSyncGroup()` — per-interface VLAN detail editing.
- Bulk operations: `initializeBulkEditApply()`, `deleteSelectedInterfaces()`.
- Table filtering: `initializeTableFilters()`, `filterTable()` — client-side row filtering.
- URL/tab state: `initializeTabs()`, `getDeviceIdFromUrl()`, `setInterfaceNameFieldFromURL()`.
- Cache countdowns: `initializeCountdown()`, `initializeCountdowns()`.
- CSRF token extracted via `document.querySelector('[name=csrfmiddlewaretoken]').value`.

View File

@@ -0,0 +1,50 @@
---
applyTo: "tests/**"
description: Testing patterns and conventions for the NetBox LibreNMS plugin
---
# Testing Patterns
## General Test Conventions
- Use plain **pytest classes**, not Django `TestCase`. Avoid `from django.test import TestCase`.
- **Never use `@pytest.mark.django_db`** for unit tests—mock all database interactions with `MagicMock`.
- Use **inline imports** inside test methods to avoid Django initialization at module load time.
- Mock NetBox models (Device, Job, User) with `MagicMock()` instead of creating real instances.
- Use `assert x == y` syntax, not `self.assertEqual(x, y)` (no TestCase inheritance).
- See [docs/development/testing.md](../../docs/development/testing.md) for test file structure and running instructions.
## Background Job Tests
- Instantiate `JobRunner` subclasses using `object.__new__(JobClass)` to bypass `__init__`, then set `job.job = MagicMock()` and `job.logger = MagicMock()`. See `create_mock_job_runner()` helper in `tests/test_background_jobs.py`.
- Patch deferred/inline imports at their **source** module (e.g., `netbox_librenms_plugin.import_utils.process_device_filters`), not the consuming module.
- Patch `cache` where imported: `netbox_librenms_plugin.views.imports.list.cache`, not `django.core.cache.cache`.
- Test view decision logic by setting `view._filter_form_data = {...}` directly, not via HTTP requests.
- **Never use `RequestFactory`**—mock request objects directly or test method logic in isolation.
- Cache key tests must patch `get_validated_device_cache_key` from `import_utils.py`; never hardcode key formats like `job_123_device_1`.
## Test File Naming
- Follow the `test_{module_name}.py` convention for new test files.
- `test_netbox_librenms_plugin.py` is an empty placeholder — do not add tests there.
## Test Coverage by Module
- `librenms_api.py``test_librenms_api.py`, `test_librenms_api_helpers.py`
- `import_utils.py`, `import_validation_helpers.py`, `utils.py``test_import_utils.py`, `test_import_validation_helpers.py`, `test_utils.py`
- `jobs.py`, `views/imports/list.py``test_background_jobs.py`
- Permission mixins, API permissions, constants → `test_permissions.py`
- VLAN API, mode detection, comparison, sync → `test_vlan_sync.py`
- `VlanAssignmentMixin`, VLAN enrichment → `test_interface_vlan_sync.py`
- Views (`views/sync/`, `views/object_sync/`, `views/imports/actions.py`) — no dedicated test files yet. Test business logic via the utility modules they call, not via HTTP requests.
## Permission Test Patterns
When testing permissions (see `test_permissions.py` for reference):
- Create a mock view instance with `object.__new__(ViewClass)`, set `request = MagicMock()` with `request.user.has_perm.side_effect = lambda p: p in allowed_perms`.
- For `NetBoxObjectPermissionMixin` tests, set `required_object_permissions` on the instance before calling `check_object_permissions()`.
- Test both the individual methods (`has_write_permission()`, `check_object_permissions()`) and the combined `require_all_permissions()` flow.
- For JSON variants (`require_all_permissions_json`), assert `isinstance(response, JsonResponse)` and check `response.status_code == 403`.
## Shared Fixtures (`conftest.py`)
Reuse fixtures from `tests/conftest.py` instead of creating ad-hoc mocks:
- **Configuration**: `mock_multi_server_config`, `mock_legacy_config`
- **API client**: `mock_librenms_api`
- **NetBox objects**: `mock_netbox_device`, `mock_netbox_vm`, `mock_netbox_site`, `mock_netbox_platform`, `mock_netbox_device_type`, `mock_netbox_device_role`, `mock_netbox_cluster`, `mock_netbox_rack`
- **HTTP responses**: `mock_response_factory`, `mock_success_response`, `mock_device_response`, `mock_error_response`, `mock_auth_error_response`
- **Import workflow**: `sample_librenms_device`, `sample_librenms_device_minimal`, `sample_validation_state`, `sample_validation_state_vm`

49
.github/pull_request_template.md vendored Normal file
View File

@@ -0,0 +1,49 @@
## Summary
Briefly describe what this PR does in plain English, and provide as much of the following information as possible.
## Motivation / Problem
What issue does this solve?
- Bug
- Feature
- Refactor
- Maintenance / cleanup
Link any related issues if applicable.
## Scope of Change
Delete items that dont apply:
- Sync/Import logic
- NetBox models / ORM
- LibreNMS API interaction
- Config / settings
- Web UI / templates
- Database migrations
- Tests
- Docs only
- Other: <describe>
## How Was This Tested?
Delete items that dont apply and describe briefly.
- Unit tests: <yes/no + what>
- Manual testing: <yes/no + what>
- Not tested: <explain why>
### Manual Test Steps (if applicable)
1.
2.
3.
## Risk Assessment
- Does this change affect existing users?
- Could this cause unintended imports / updates?
Explain briefly.
## Backwards Compatibility
- No breaking changes
- Breaking change (explain and document)
## Other Notes
Anything the maintainer(s) should pay particular attention to?

103
.github/workflows/codeql.yml vendored Normal file
View File

@@ -0,0 +1,103 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL Advanced"
on:
push:
branches: [ "master", "develop" ]
pull_request:
branches: [ "master", "develop" ]
schedule:
- cron: '35 13 * * 0'
jobs:
analyze:
name: Analyze (${{ matrix.language }})
# Runner size impacts CodeQL analysis time. To learn more, please see:
# - https://gh.io/recommended-hardware-resources-for-running-codeql
# - https://gh.io/supported-runners-and-hardware-resources
# - https://gh.io/using-larger-runners (GitHub.com only)
# Consider using larger runners or machines with greater resources for possible analysis time improvements.
runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
permissions:
# required for all workflows
security-events: write
# required to fetch internal or private CodeQL packs
packages: read
# only required for workflows in private repositories
actions: read
contents: read
strategy:
fail-fast: false
matrix:
include:
- language: actions
build-mode: none
- language: javascript-typescript
build-mode: none
- language: python
build-mode: none
# CodeQL supports the following values keywords for 'language': 'actions', 'c-cpp', 'csharp', 'go', 'java-kotlin', 'javascript-typescript', 'python', 'ruby', 'rust', 'swift'
# Use `c-cpp` to analyze code written in C, C++ or both
# Use 'java-kotlin' to analyze code written in Java, Kotlin or both
# Use 'javascript-typescript' to analyze code written in JavaScript, TypeScript or both
# To learn more about changing the languages that are analyzed or customizing the build mode for your analysis,
# see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.
# If you are analyzing a compiled language, you can modify the 'build-mode' for that language to customize how
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
steps:
- name: Checkout repository
uses: actions/checkout@v6
# Add any setup steps before running the `github/codeql-action/init` action.
# This includes steps like installing compilers or runtimes (`actions/setup-node`
# or others). This is typically only required for manual builds.
# - name: Setup runtime (example)
# uses: actions/setup-example@v1
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v4
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality
# If the analyze step fails for one of the languages you are analyzing with
# "We were unable to automatically build your code", modify the matrix above
# to set the build mode to "manual" for that language. Then modify this step
# to build your code.
# Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
- name: Run manual build steps
if: matrix.build-mode == 'manual'
shell: bash
run: |
echo 'If you are using a "manual" build mode for one or more of the' \
'languages you are analyzing, replace this with the commands to build' \
'your code, for example:'
echo ' make bootstrap'
echo ' make release'
exit 1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v4
with:
category: "/language:${{matrix.language}}"

50
.github/workflows/lint-format.yaml vendored Normal file
View File

@@ -0,0 +1,50 @@
name: Lint and Format
on:
push:
branches:
- master
- develop
pull_request:
branches:
- master
- develop
jobs:
format-and-lint:
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: '3.9'
cache: 'pip'
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install ruff
- name: Run Ruff linting
run: |
echo "::group::Ruff Linting"
ruff check . --output-format=github
echo "::endgroup::"
- name: Run Ruff formatting check
run: |
echo "::group::Ruff Formatting"
ruff format --check .
echo "::endgroup::"
- name: Report formatting issues
if: failure()
run: |
echo "::error::Formatting or linting issues detected!"
echo "To fix locally, run:"
echo " ruff check --fix ."
echo " ruff format ."
echo "Then commit and push the changes."

18
.github/workflows/mkdocs.yaml vendored Normal file
View File

@@ -0,0 +1,18 @@
name: ci
on:
push:
branches:
- master
- main
permissions:
contents: write
jobs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: 3.x
- run: pip install mkdocs-material mkdocs-autorefs mkdocs-material-extensions mkdocstrings mkdocstrings-python-legacy mkdocs-include-markdown-plugin
- run: mkdocs gh-deploy --force

58
.github/workflows/publish-pypi.yaml vendored Normal file
View File

@@ -0,0 +1,58 @@
# see: https://packaging.python.org/en/latest/guides/publishing-package-distribution-releases-using-github-actions-ci-cd-workflows/
name: Publish Python 🐍 distribution 📦 to PyPI
on:
push:
branches:
- master
tags:
- '*'
jobs:
build:
name: Build distribution 📦
runs-on: ubuntu-latest
steps:
- name: Clean previous builds
run: rm -rf dist/
- uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
- name: Set up Python
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: "3.x"
- name: Install pypa/build
run: >-
python3 -m
pip install
build
--user
- name: Build a binary wheel and a source tarball
run: python3 -m build
- name: Store the distribution packages
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: python-package-distributions
path: dist/
publish-to-pypi:
name: >-
Publish Python 🐍 distribution 📦 to PyPI
if: startsWith(github.ref, 'refs/tags/') # only publish to PyPI on tag pushes
needs:
- build
runs-on: ubuntu-latest
environment:
name: pypi
url: https://pypi.org/p/netbox-librenms-plugin
permissions:
id-token: write # IMPORTANT: mandatory for trusted publishing
steps:
- name: Download all the dists
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: python-package-distributions
path: dist/
- name: Publish distribution 📦 to PyPI
uses: pypa/gh-action-pypi-publish@cef221092ed1bacb1cc03d23a2d87d1d172e277b # release/v1

88
.github/workflows/test.yaml vendored Normal file
View File

@@ -0,0 +1,88 @@
name: Test with all supported NetBox versions
on:
push:
branches:
- master
- develop
pull_request:
branches:
- master
- develop
jobs:
test-netbox:
runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.12", "3.13", "3.14"]
services:
redis:
image: redis
ports:
- 6379:6379
postgres:
image: postgres
env:
POSTGRES_USER: netbox
POSTGRES_PASSWORD: netbox
options: >-
--health-cmd pg_isready
--health-interval 10s
--health-timeout 5s
--health-retries 5
ports:
- 5432:5432
steps:
- name: Checkout code
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
path: netbox-librenms-plugin
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@a309ff8b426b58ec0e2a45f0f869d46889d02405 # v6.2.0
with:
python-version: ${{ matrix.python-version }}
- name: Checkout NetBox
uses: actions/checkout@de0fac2e4500dabe0009e67214ff5f5447ce83dd # v6.0.2
with:
repository: "netbox-community/netbox"
path: netbox
ref: main
- name: Install NetBox LibreNMS Plugin
working-directory: netbox-librenms-plugin
run: |
pip install -e .
pip install pytest pytest-django pytest-cov
- name: Set up configuration
working-directory: netbox
run: |
ln -s "$(pwd)/../netbox-librenms-plugin/media/configuration.testing.py" netbox/netbox/configuration.py
python -m pip install --upgrade pip
python -m pip install tblib
pip install -r requirements.txt -U
- name: Run tests
working-directory: netbox/netbox
env:
NETBOX_CONFIGURATION: netbox.configuration
run: |
python -m pytest ../../netbox-librenms-plugin/netbox_librenms_plugin/tests/ -v \
--cov=netbox_librenms_plugin \
--cov-report=html:../../netbox-librenms-plugin/coverage_html \
--cov-report=term-missing
- name: Upload coverage report
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
if: matrix.python-version == '3.12'
with:
name: coverage-report
path: netbox-librenms-plugin/coverage_html/