logagent
A simple program to scan a log file for text tags and emit actions
Science Score: 52.0%
This score indicates how likely this project is to be science-related based on various indicators:
-
✓CITATION.cff file
Found CITATION.cff file -
✓codemeta.json file
Found codemeta.json file -
✓.zenodo.json file
Found .zenodo.json file -
○DOI references
-
○Academic publication links
-
○Academic email domains
-
✓Institutional organization owner
Organization caltechlibrary has institutional domain (www.library.caltech.edu) -
○JOSS paper metadata
-
○Scientific vocabulary similarity
Low similarity (10.7%) to scientific vocabulary
Repository
A simple program to scan a log file for text tags and emit actions
Basic Info
- Host: GitHub
- Owner: caltechlibrary
- License: other
- Language: TypeScript
- Default Branch: main
- Size: 568 KB
Statistics
- Stars: 0
- Watchers: 3
- Forks: 0
- Open Issues: 1
- Releases: 4
Metadata Files
README.md
Log Agent Project
A set of tools to automate some of the adhoc log analysis we perform on RDM repositories.
LogAgent
This is a simplified log processor that looks for explicit text on a line, parses the line for an IP address and then applies the associated action. It is inspired by fail2ban but is written in response fail2ban's complexity. Caltech Library needed a simple tool to do a narrow task that was oddly challenging using fail2ban. There is always a balancing act between a tool features and those that are simpler targeting a more specific issue.
LogAnalyst
This provides a quick and dirty way of viewing aggregated counts based on transform the NginX log entries into structed data.
Approach
LogAgent and LogAnalyst read logs input line by line. LogAgent checks if a tag (explicit sub-string) is contained in that line. If a match is found then the agent extracts any IP addresses identified before applying a rule associated with the tag. LogAnalyst will look at the log entry transform it into structured data for aggregating various simple counts.
LogAgent requires a configuration file written in YAML. The configuration holds an array of objects. Each object has the following attributes. LogAnalyst just reads the log file as a stream, usually from standard input. When the processing is complete a simple analysis is displayed as a JSON object.
LogAgent Configuration
tag : The explicit search string (i.e. not regular expressions)
action : The command to execute if tag is found
Here's an example configuration YAML file.
~~~YAML - tag: BadBot action: | sudo iptables -A INPUT -p tcp -m multiport --dports 80,443 -s {ipaddress} -j DROP ~~~
If the text "BadBot" is found in the log line. and the IP address "156.59.198.136" was found in the log line then the following command would be executed.
~~~shell sudo iptables -A INPUT -p tcp -m multiport --dports 80,443 -s 156.59.198.136 -j DROP ~~~
For more information see the following documentation pages.
- User Manual
- Installation and compiling Log Agent from source
- LICENSE
- About Log Agent
- Cite with CITATION.cff
Owner
- Name: Caltech Library
- Login: caltechlibrary
- Kind: organization
- Email: helpdesk@library.caltech.edu
- Location: Pasadena, CA 91125
- Website: https://www.library.caltech.edu/
- Repositories: 84
- Profile: https://github.com/caltechlibrary
We manage the physical and digital holdings of the California Institute of Technology, provide services and training, and develop open-source software.
Citation (CITATION.cff)
cff-version: 1.2.0
message: "If you use this software, please cite it as below."
type: software
title: logagent
abstract: "LogAgent was designed detecting bot based on log entries searching for specific sub strings. LogAnalyst was written to aggregate the structure in our logs to provide simple count
relations between log fields. The are intended to work with the NginX access logs we generate from our RDM instances. The tooling may work with other types of logs."
authors:
- family-names: Doiel
given-names: R. S.
orcid: https://orcid.org/0000-0003-0900-6903
email: rsdoiel@caltech.edu
repository-code: "git+https://github.com/caltechlibrary/logagent"
version: 0.0.4
date-released: 2024-03-26
license-url: "https://data.caltech.edu/license"
keywords:
- logging
- agents
- analyzer
CodeMeta (codemeta.json)
{
"@context": "https://w3id.org/codemeta/3.0",
"type": "SoftwareSourceCode",
"codeRepository": "git+https://github.com/caltechlibrary/logagent",
"author": [
{
"id": "https://orcid.org/0000-0003-0900-6903",
"type": "Person",
"givenName": "R. S.",
"familyName": "Doiel",
"affiliation": {
"type": "Organization",
"name": "Caltech Library, California Institute of Technology"
},
"email": "rsdoiel@caltech.edu"
}
],
"dateCreated": "2025-01-23",
"dateModified": "2025-03-26",
"datePublished": "2024-03-26",
"description": "LogAgent was designed detecting bot based on log entries searching for specific sub strings. LogAnalyst was written to aggregate the structure in our logs to provide simple count \nrelations between log fields. The are intended to work with the NginX access logs we generate from our RDM instances. The tooling may work with other types of logs.",
"funder": {
"type": "Organization",
"name": "Caltech Library"
},
"keywords": [
"logging",
"agents",
"analyzer"
],
"name": "logagent",
"license": "https://data.caltech.edu/license",
"operatingSystem": [
"Linux",
"Windows",
"macOS"
],
"programmingLanguage": [
"TypeScript"
],
"runtimePlatform": "Deno",
"softwareRequirements": [
"deno >= 2.2.5"
],
"version": "0.0.4",
"developmentStatus": "wip",
"issueTracker": "https://github.com/caltechlibrary/logagent/issues",
"downloadUrl": "https://github.com/caltechlibrary/logagent/releases",
"releaseNotes": "Working proof of concept fo LogAgent and LogAnalyst."
}GitHub Events
Total
- Release event: 3
- Public event: 1
- Push event: 21
- Create event: 4
Last Year
- Release event: 3
- Public event: 1
- Push event: 21
- Create event: 4