mirror of
https://github.com/h9zdev/GeoSentinel
synced 2026-04-25 17:25:10 +02:00
Merge branch 'main' into main
This commit is contained in:
H9
258
README.md
258
README.md
@@ -1,24 +1,44 @@
|
||||
# 🌍 GeoSentinel
|
||||
|
||||
<div align="center">
|
||||
|
||||
[](https://www.python.org/)
|
||||
[](https://www.linux.org/)
|
||||
[](https://github.com/h9zdev/GeoSentinel)
|
||||
[](https://github.com/h9zdev/GeoSentinel)
|
||||
|
||||
</div>
|
||||
|
||||
<p align="center">
|
||||
<img src="https://raw.githubusercontent.com/h9zdev/GeoSentinel/main/images/GeoSentinel.png" alt="GeoSentinel" />
|
||||
</p>
|
||||
|
||||
|
||||
**GeoSentinel** is a geospatial monitoring platform that tracks global movement in real time.
|
||||
|
||||
It aggregates ship and flight routes, live coordinates, and geodata into a unified system, providing clear geographic and geopolitical awareness for analysis, visualization, and decision-making.
|
||||
. 🚀
|
||||
|
||||
🚀 Visit Blog: [VaradScript.web.app](https://varadaraj.online/)
|
||||
🚀 Visit Blog: [haybnz.web.app](https://haybnz.web.app/blog)
|
||||
🚀 **Visit the Blog:**
|
||||
👉 https://haybnz.web.app/blog
|
||||
|
||||
> 🔴 **NOTE:**
|
||||
> <span style="color:red;">Get Updates on the Latest Geo Sentinel AI Releases</span>
|
||||
>
|
||||
> 👉 Subscribe here:
|
||||
> https://docs.google.com/forms/d/e/1FAIpQLSe3qBh6r1orih2MkLf5DjdolX0jv5Abct02363lLxpXEute-Q/viewform?usp=header
|
||||
🚀 **Visit the Blog:**
|
||||
👉 https://varadaraj.online/
|
||||
|
||||
|
||||
---
|
||||
|
||||
🔴 **NOTE**
|
||||
Stay updated with the latest **Geo Sentinel AI** releases and announcements.
|
||||
|
||||
👉 **Subscribe here:**
|
||||
https://docs.google.com/forms/d/e/1FAIpQLSe3qBh6r1orih2MkLf5DjdolX0jv5Abct02363lLxpXEute-Q/viewform
|
||||
|
||||
## 📚 Documentation
|
||||
|
||||
Detailed guides on how to use GeoSentinel's advanced features:
|
||||
- 🤖 [GeoSential AI Guide](docs/geosential_ai.md) - Learn how to use the AI assistant and its commands.
|
||||
- 🔍 [Search & Dark Web Guide](docs/search_options.md) - Instructions for OSINT and dark web searching.
|
||||
|
||||
## 🌟 Features
|
||||
|
||||
- 🗺️ Access to GeoJSON data and surveillance grid tiles.
|
||||
@@ -29,19 +49,44 @@ It aggregates ship and flight routes, live coordinates, and geodata into a unifi
|
||||
- 📰 Geopolitical news and sentiment analysis.
|
||||
- 💹 Market data for commodities and cryptocurrencies.
|
||||
- 🌐 Translation services.
|
||||
-
|
||||
- 🔒 TOR integration for enhanced privacy.
|
||||
- 🤖 OLLAMA AI integration for local LLM processing.
|
||||
- 🕵️♂️ **Darkweb Search**: Anonymous searching across multiple .onion engines via TOR integration.
|
||||
- 🔍 **Advanced Web Scraper**: Multi-engine OSINT search with Google Dorking for social media platforms (Twitter, Reddit, Instagram, etc.).
|
||||
- 🤖 **GeoSential AI**: Intelligent assistant for automated real-time tracking of flights and vessels with integrated OSINT.
|
||||
|
||||
### 🌍 Earth HTML Features
|
||||
- Interactive global map with real-time tracking
|
||||
- Advanced search capabilities (HEX, flight, vessel, coordinates)
|
||||
- TomTom Maps API integration for detailed mapping
|
||||
- Activity logging and user tracking
|
||||
- Responsive design for all devices
|
||||
- GPS metadata extraction from images
|
||||
- Real-time data visualization
|
||||
- Integrated GeoSential AI for automated tracking and analysis
|
||||
- Advanced web scanning with social media dorking capabilities
|
||||
|
||||
### 📦 Download and Move `geodata` Folder to Root Directory
|
||||
|
||||
### 🔗 Download Link
|
||||
👉 [Download geodata folder](https://drive.proton.me/urls/RJB7K8HXTM#r7PnpGiVkg6P)
|
||||
|
||||
### 🔗 Download Link
|
||||
-ADD API KEY IN app.py
|
||||
-ADD API KEY IN earth.html {on line1850 const tomtomApiKey = 'ADD_API+KEY';}
|
||||
## 📝 Configuration
|
||||
|
||||
### API Keys Required
|
||||
- **TomTom Maps API**: Add your key in `templates/earth.html` (line ~1850)
|
||||
```javascript
|
||||
const tomtomApiKey = 'YOUR_TOMTOM_API_KEY';
|
||||
```
|
||||
|
||||
- **Other APIs**: Add relevant API keys in `app.py`
|
||||
|
||||
---
|
||||
|
||||
|
||||
# 🛰️ GeoSentinel Installation Guide
|
||||
|
||||
## 📥 Clone or Fork the Repository
|
||||
|
||||
|
||||
### Option 1: Clone directly
|
||||
```bash
|
||||
@@ -91,108 +136,127 @@ python app.py
|
||||
- Earth View
|
||||
https://127.0.0.1:8000/earth
|
||||
|
||||
- News Dashboard
|
||||
- News Dashboard
|
||||
https://127.0.0.1:8000/news
|
||||
|
||||
---
|
||||
|
||||
## ✅ Notes
|
||||
- SSL warnings are normal for localhost with HTTPS.
|
||||
- Use http://127.0.0.1:8000 if HTTPS is not configured.
|
||||
## ⚙️ API Endpoints
|
||||
|
||||
### 🌎 Earth
|
||||
|
||||
- **GET /earth**
|
||||
- Renders the main earth page.
|
||||
|
||||
### 🗺️ GeoJSON
|
||||
|
||||
- **GET /api/geojson/<filename>**
|
||||
- Retrieves a summary of a GeoJSON file.
|
||||
- Example: `/api/geojson/example.geojson`
|
||||
|
||||
### 🛰️ Surveillance Grid
|
||||
|
||||
- **GET /api/geo/index**
|
||||
- Retrieves the surveillance grid index.
|
||||
- **GET /api/geo/tile/<z>/<x>/<y>**
|
||||
- Retrieves a specific surveillance grid tile.
|
||||
- Example: `/api/geo/tile/1/2/3`
|
||||
|
||||
### ✈️ Flights
|
||||
|
||||
- **GET /api/geo/flights**
|
||||
- Fetches live flight data.
|
||||
- **GET /api/geo/flight/meta/<callsign>**
|
||||
- Retrieves metadata for a specific flight.
|
||||
- Example: `/api/geo/flight/meta/UAL123`
|
||||
|
||||
### 🚢 Vessels
|
||||
|
||||
- **GET /api/geo/vessels**
|
||||
- Fetches live vessel data.
|
||||
- **GET /api/geo/vessel/path/<mmsi>**
|
||||
- Retrieves the historical path of a vessel.
|
||||
- Example: `/api/geo/vessel/path/123456789`
|
||||
|
||||
### 📸 Image Analysis
|
||||
|
||||
- **POST /api/geo/segment**
|
||||
- Performs aerial segmentation on a satellite tile.
|
||||
- **POST /api/geo/analyze-upload** or **/upload**
|
||||
- Analyzes an uploaded image for objects and GPS metadata.
|
||||
|
||||
### 📰 News
|
||||
|
||||
- **GET /api/geo/news**
|
||||
- Fetches geopolitical news for a specific location.
|
||||
- **POST /api/news/analyze**
|
||||
- Analyzes the sentiment of a news article.
|
||||
- **GET /api/news/advanced**
|
||||
- Performs an advanced search for news articles.
|
||||
|
||||
### 💹 Market
|
||||
|
||||
- **GET /api/market/data**
|
||||
- Fetches market data for commodities and cryptocurrencies.
|
||||
|
||||
### 🌐 Translate
|
||||
|
||||
- **GET /api/translate**
|
||||
- Translates text to English.
|
||||
|
||||
## 🚀 How to Use
|
||||
|
||||
## 🔒 TOR Installation & Setup (Linux)
|
||||
|
||||
### Option 1: Install TOR from Package Manager
|
||||
```bash
|
||||
curl http://localhost:8000/api/geo/flights
|
||||
# Ubuntu/Debian
|
||||
sudo apt-get update
|
||||
sudo apt-get install tor torbrowser-launcher -y
|
||||
|
||||
# Fedora/RHEL
|
||||
sudo dnf install tor torbrowser-launcher -y
|
||||
```
|
||||
|
||||
### Option 2: Install TOR from Source
|
||||
```bash
|
||||
# Download TOR
|
||||
cd /tmp
|
||||
wget https://archive.torproject.org/tor-package-archive/tor-latest.tar.gz
|
||||
tar -xzf tor-latest.tar.gz
|
||||
cd tor-*
|
||||
|
||||
# Compile and install
|
||||
./configure
|
||||
make
|
||||
sudo make install
|
||||
```
|
||||
|
||||
### Start TOR Service
|
||||
```bash
|
||||
# Start TOR daemon
|
||||
sudo systemctl start tor
|
||||
sudo systemctl enable tor # Enable on boot
|
||||
|
||||
# Or run manually
|
||||
tor
|
||||
|
||||
# Verify TOR is running
|
||||
curl --socks5 127.0.0.1:9050 https://check.torproject.org/api/ip
|
||||
```
|
||||
|
||||
|
||||
|
||||
## 🤖 OLLAMA Installation & Setup
|
||||
|
||||
### Installation Steps
|
||||
|
||||
#### Step 1: Download OLLAMA
|
||||
```bash
|
||||
# macOS
|
||||
curl -fsSL https://ollama.ai/install.sh | sh
|
||||
|
||||
# Linux
|
||||
curl -fsSL https://ollama.ai/install.sh | sh
|
||||
|
||||
# Or download from
|
||||
https://ollama.ai/download
|
||||
```
|
||||
|
||||
#### Step 2: Verify Installation
|
||||
```bash
|
||||
ollama --version
|
||||
```
|
||||
|
||||
#### Step 3: Pull a Model
|
||||
```bash
|
||||
# Pull Llama 2 model (7B parameters)
|
||||
ollama pull llama2
|
||||
|
||||
# Or pull other models
|
||||
ollama pull mistral # Mistral model
|
||||
ollama pull neural-chat # Neural Chat model
|
||||
ollama pull orca-mini # Orca Mini model
|
||||
```
|
||||
|
||||
#### Step 4: Run OLLAMA Server
|
||||
```bash
|
||||
# Start OLLAMA server (runs on localhost:11434)
|
||||
ollama serve
|
||||
```
|
||||
|
||||
## ▶️ Watch on YouTube
|
||||
|
||||
[](https://youtu.be/v4IKzEb03u0)
|
||||
|
||||
🔗 Direct link: https://youtu.be/v4IKzEb03u0
|
||||
|
||||
|
||||
|
||||
## 🙏 API TO USE
|
||||
|
||||
- [OpenStreetMap](https://www.openstreetmap.org/)
|
||||
- [ADSB.one](https://adsb.one/)
|
||||
- [AISstream.io](https://aisstream.io/)
|
||||
- [CoinGecko](https://www.coingecko.com/)
|
||||
- [NewsAPI](https://newsapi.org/)
|
||||
- [tomtom](https://developer.tomtom.com)
|
||||
Below is a list of APIs used in GeoSentinel and where to obtain your API keys:
|
||||
|
||||
- 🌍 [OpenStreetMap](https://www.openstreetmap.org/) - Geocoding and map data.
|
||||
- ✈️ [ADSB.one](https://adsb.one/) - Real-time flight tracking data.
|
||||
- 🚢 [AISstream.io](https://aisstream.io/) - Real-time vessel tracking (AIS).
|
||||
- 📊 [CoinGecko](https://www.coingecko.com/) - Cryptocurrency market data.
|
||||
- 📰 [NewsAPI](https://newsapi.org/) - Global news feeds.
|
||||
- 🗺️ [TomTom Maps](https://developer.tomtom.com) - Mapping and routing services.
|
||||
- 📶 [OpenCellID](https://opencellid.org/) - Cell tower location data.
|
||||
- 🤗 [Hugging Face](https://huggingface.co/) - AI models and inference.
|
||||
- 🤖 [OpenRouter](https://openrouter.ai/) - Unified AI model gateway.
|
||||
- 📡 [OpenSky Network](https://opensky-network.org/) - Flight metadata and routing.
|
||||
- 🌐 [MyMemory Translation](https://mymemory.translated.net/) - Free translation services.
|
||||
- 🐦 [Twitter/X API](https://developer.twitter.com/) - Social media OSINT.
|
||||
- 🔍 [DuckDuckGo](https://duckduckgo.com/) - Web search integration.
|
||||
- 🌐 [Google Search](https://www.google.com/) - Web search integration.
|
||||
- 🅱️ [Bing Search](https://www.bing.com/) - Web search integration.
|
||||
- 🔒 [Ahmia](https://ahmia.fi/) - Dark web search services.
|
||||
- 🦙 [Ollama](https://ollama.com/) - Local LLM processing.
|
||||
|
||||
## 🗺️ Images of GeoSentinel UI
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
57
docs/geosential_ai.md
Normal file
57
docs/geosential_ai.md
Normal file
@@ -0,0 +1,57 @@
|
||||
# 🤖 GeoSential AI Documentation
|
||||
|
||||
Welcome to the **GeoSential AI** manual! GeoSential AI is your high-tech Geospatial Intelligence (GEOINT) and OSINT assistant, designed to automate tracking and provide real-time global briefings. 🌍🛰️
|
||||
|
||||
## 🚀 Overview
|
||||
|
||||
GeoSential AI integrates real-time web data, map functionality, and semantic memory to assist in monitoring global activities. It can track flights, vessels, scan sectors for signals, and provide up-to-the-minute news and market data.
|
||||
|
||||
## 🛠️ How to Use
|
||||
|
||||
### 🖥️ Interface
|
||||
You can interact with GeoSential AI directly through the **Earth View** dashboard in the GeoSentinel UI. Look for the AI chat panel to start a conversation.
|
||||
|
||||
### 🔌 API Endpoint
|
||||
For developers, the AI is accessible via the following endpoint:
|
||||
- **URL**: `/api/geosentialai/chat`
|
||||
- **Method**: `POST`
|
||||
- **Payload**:
|
||||
```json
|
||||
{
|
||||
"message": "Track flight UAE202",
|
||||
"web_search": true,
|
||||
"human_mode": false,
|
||||
"engine": "huggingface",
|
||||
"context": {}
|
||||
}
|
||||
```
|
||||
|
||||
## 📜 Commands & Tags
|
||||
|
||||
GeoSential AI can trigger specific actions in the UI by outputting special tags. These tags allow the AI to interact directly with the map:
|
||||
|
||||
| Command | Action |
|
||||
| :--- | :--- |
|
||||
| `[TRACK_FLIGHT: <icao>]` | ✈️ Zooms the map to a specific flight using its ICAO hex code. |
|
||||
| `[TRACK_VESSEL: <mmsi>]` | 🚢 Zooms the map to a specific vessel using its MMSI number. |
|
||||
| `[SHOW_WEATHER: <lat>, <lng>]` | 🌦️ Opens the meteorology/environment GUI for the specified coordinates. |
|
||||
| `[SCAN_MAP: <lat>, <lng>]` | 📡 Zooms to coordinates and initiates a sector-wide signal scan. |
|
||||
|
||||
## 🔍 Search Integration
|
||||
|
||||
GeoSential AI is equipped with a powerful web search capability 🕵️♂️.
|
||||
- **Auto-Trigger**: The AI automatically enables web search if your message contains keywords like *news, stock, price, market, update, latest,* or *happening*.
|
||||
- **DuckDuckGo Integration**: It uses DuckDuckGo to pull real-time snippets from the web to ground its answers in current events.
|
||||
|
||||
## 🧠 Memory System
|
||||
|
||||
The AI uses **ChromaDB** to maintain a "Memory Stream" 🧠. It remembers previous interactions and can recall relevant context to provide more personalized and consistent assistance.
|
||||
|
||||
## ⚙️ AI Engines
|
||||
|
||||
You can choose between two processing engines:
|
||||
1. **Cloud (Hugging Face)**: Uses `Llama-3.1-8B-Instruct` for high-performance reasoning (Default).
|
||||
2. **Local (Ollama)**: Uses the `phi` model running locally on your machine for enhanced privacy and offline use.
|
||||
|
||||
---
|
||||
*Stay informed, stay ahead with GeoSential AI. 🚀*
|
||||
49
docs/search_options.md
Normal file
49
docs/search_options.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# 🔍 Search Options & Dark Web Documentation
|
||||
|
||||
Explore the powerful search capabilities of GeoSentinel, from surface web OSINT to the depths of the dark web. 🕵️♂️🔒
|
||||
|
||||
## 🌐 Advanced Web Scan
|
||||
|
||||
The Advanced Web Scan tool allows you to perform deep OSINT searches across multiple platforms and search engines simultaneously.
|
||||
|
||||
### 🛠️ How to Use
|
||||
- **URL**: `/api/tools/web_scan`
|
||||
- **Method**: `POST`
|
||||
- **Parameters**:
|
||||
- `query`: Your search term or target.
|
||||
- `type`: `text`, `images`, or `all`.
|
||||
- `sources`: A list of platforms to target.
|
||||
- `aggressive`: Set to `true` for deep scraping of result pages.
|
||||
|
||||
### 📱 Supported Sources
|
||||
You can restrict your search to specific high-value OSINT sources:
|
||||
- **Social Media**: Twitter, Reddit, Instagram, LinkedIn.
|
||||
- **Communication**: Telegram, Discord.
|
||||
- **Developer/Code**: GitHub, StackOverflow.
|
||||
- **Leaks/Dumps**: Pastebin, BreachForums.
|
||||
- **General Web**: Multi-engine aggregation (Google, Bing, DuckDuckGo).
|
||||
|
||||
### 🚀 Aggressive Mode
|
||||
When **Aggressive Mode** is enabled, GeoSentinel goes beyond simple search results. It attempts to visit the top result links and extract the actual page content to provide a more comprehensive intelligence briefing.
|
||||
|
||||
---
|
||||
|
||||
## 🔒 Dark Web Search
|
||||
|
||||
GeoSentinel provides integrated access to the dark web, allowing you to search across multiple `.onion` search engines anonymously. 🕶️
|
||||
|
||||
### 🛡️ How it Works
|
||||
1. **Tor Network**: If a Tor proxy is running locally (port 9050), GeoSentinel routes queries through the Tor network to access `.onion` sites directly.
|
||||
2. **Onion Aggregation**: It queries a vast list of dark web engines, including:
|
||||
- Ahmia
|
||||
- OnionLand
|
||||
- Torgle
|
||||
- Torch
|
||||
- ...and many others.
|
||||
3. **Clearnet Fallback**: If Tor is not available locally, the system automatically falls back to the Ahmia clearnet proxy to fetch results.
|
||||
|
||||
### 🕵️♂️ Accessing Dark Web Results
|
||||
Results from the dark web are flagged with the `TOR_NETWORK` or `Ahmia_Clearnet` source tag. These links typically end in `.onion` and require a Tor-enabled browser (like Tor Browser) to open.
|
||||
|
||||
---
|
||||
*Unlock the hidden layers of the web with GeoSentinel. 🔍🔒*
|
||||
@@ -5,3 +5,7 @@ feedparser
|
||||
numpy
|
||||
opencv-python-headless
|
||||
Pillow
|
||||
ollama
|
||||
gTTS
|
||||
pysocks
|
||||
stem
|
||||
|
||||
1743
templates/earth.html
1743
templates/earth.html
File diff suppressed because it is too large
Load Diff
33
verify_search.py
Normal file
33
verify_search.py
Normal file
@@ -0,0 +1,33 @@
|
||||
from duckduckgo_search import DDGS
|
||||
|
||||
print("Verifying Image Search (SafeSearch OFF)...")
|
||||
try:
|
||||
ddgs = DDGS()
|
||||
# "porn" query to test safe search is off
|
||||
results = ddgs.images("porn", max_results=5, safesearch='off')
|
||||
if results:
|
||||
print(f"✅ Image search returned {len(results)} results.")
|
||||
print(f"First result title: {results[0].get('title')}")
|
||||
else:
|
||||
print("❌ Image search returned NO results.")
|
||||
except Exception as e:
|
||||
print(f"❌ Image search failed: {e}")
|
||||
|
||||
print("\nVerifying Video Search (SafeSearch OFF + Thumbnails)...")
|
||||
try:
|
||||
results = ddgs.videos("test video", max_results=5, safesearch='off')
|
||||
if results:
|
||||
print(f"✅ Video search returned {len(results)} results.")
|
||||
first = results[0]
|
||||
# Check for images/thumbnail structure
|
||||
thumb = first.get('images', {}).get('large')
|
||||
if thumb:
|
||||
print(f"✅ Video thumbnail found: {thumb}")
|
||||
print(f"First result title: {first.get('title')}")
|
||||
else:
|
||||
print("❌ Video thumbnail NOT found in first result.")
|
||||
print(f"Result keys: {first.keys()}")
|
||||
else:
|
||||
print("❌ Video search returned NO results.")
|
||||
except Exception as e:
|
||||
print(f"❌ Video search failed: {e}")
|
||||
Reference in New Issue
Block a user