mirror of https://github.com/SHOGGOTH-SECTOR/sica-fondt.git synced 2026-08-01 08:30:20 +00:00

Consolidate 22 sibling repos into layered organism structure

Place useful parts of the surrounding repos into sica-fondt by layer, per the
body model (Ada = membrane; brain/endocrine/capabilities/knowledge non-Ada):

- brain/        LLM reasoning + providers (dapr, hermes, MoMoA)
- capabilities/ REPRAG sidecars: hermes tools/skills, dapr tools, parallel
                dispatch, A51 channels, and the OSINT cluster
- knowledge/    LORAG corpus: 754 cyber-skills, agency personas, secure-coding,
                MITRE ATT&CK data
- reference/    defensive threat-reference (C3, shhbruh doc) + AdaYaml parser

License handling: AGPL sources (worldosint, advanced_evolution, mercury,
Reticulum) and GPL DeTTECT are SPEC-only clean-room/port descriptions — no
copyleft code copied. MIT/Apache/data parts copied as working trees.

Safety: shhbruh escape/persistence material and C3 covert-C2 kept as reference
only, not wired into the running organism. See CONSOLIDATION.md.

https://claude.ai/code/session_01UehUqEXXJJCsHoA4voCU5c

2026-06-10 06:53:01 +00:00

2.3 KiB

Raw Blame History

API Reference — Detecting Shadow IT Cloud Usage

Libraries Used

pandas: DataFrame aggregation for traffic analysis — groupby, agg, nunique
tldextract: Accurate registered domain extraction from URLs/hostnames
csv: CSV log parsing with DictReader
re: Regex parsing for Squid proxy and BIND DNS query log formats

CLI Interface

python agent.py access.log --type proxy parse
python agent.py access.log --type proxy analyze
python agent.py dns-queries.log --type dns full
python agent.py traffic.csv --type csv --approved approved.txt full

Core Functions

`parse_proxy_log(filepath)` — Parse Squid/common proxy access logs

Regex pattern matches Squid format: timestamp duration client_ip status bytes method url. Falls back to Apache Common Log Format parsing.

`parse_dns_log(filepath)` — Parse BIND/named DNS query logs

Extracts query name and type from query: DOMAIN IN TYPE patterns. Strips trailing dots from FQDNs.

`parse_csv_log(filepath)` — Parse generic CSV traffic logs

Expects columns: timestamp, src_ip, dst_domain, bytes_out, bytes_in.

`analyze_traffic(records)` — Aggregate and classify traffic

Uses pandas groupby on domain: total_bytes (sum), request_count (count), unique_users (nunique). Falls back to collections.defaultdict if pandas unavailable.

`classify_domain(domain)` — Categorize against SaaS database

Categories: storage, email, dev_tools, ai_ml, messaging, file_sharing, vpn_proxy.

`full_audit(log_path, log_type, approved_list)` — Complete shadow IT audit

Risk Scoring

Factor	Points
Unapproved domain	+30
Storage/file-sharing/VPN category	+25
Email category	+15
Data volume (per 10 MB)	+1 (max 20)
Unique users (per user)	+3 (max 15)

SaaS Category Database

Category	Example Domains
storage	dropbox.com, box.com, mega.nz, wetransfer.com
email	protonmail.com, tutanota.com, guerrillamail.com
dev_tools	github.com, gitlab.com, replit.com
ai_ml	chat.openai.com, claude.ai, huggingface.co
messaging	telegram.org, discord.com, signal.org
file_sharing	pastebin.com, file.io, gofile.io
vpn_proxy	nordvpn.com, expressvpn.com, protonvpn.com

Dependencies

pandas >= 1.5.0
tldextract >= 3.4.0 (optional, improves domain extraction accuracy)

2.3 KiB Raw Blame History