I haven’t been very active in delving into “AI,” but recently I got some motivation to finally start playing around with Large Language Models (LLMs). I was particularly interested in running them locally and quickly discovered the common methods to improve the performance of smaller models.

Based on what I learned, I wanted to test something like this:

  1. Take a small model that my hardware can run (e.g. llama3.2:1b or llama3.2:3b).
  2. Get some data from the internet or my local files.
  3. Index the data.
  4. Provide some portion of the indexed data as context for prompts.

I’m sure there are thousands of code snippets that already do this and much better, but I wanted to tinker with it myself for better learning. This is not “how to” post, but might give some ideas to do something interesting.

Scripts used in this post can be found in this GitHub repository. Please note that the content is likely to change, as it is primarily meant for my own testing purposes.

I won’t cover setting up things lika Ollama here, but you can find more about my setup in the mentioned repository. More comprehensive guides can be found from multiple places. For example, check Ollama’s docs.

Terminology

Some relevant terminology for this post.

Context window

The context window (or “context length”) of a large language model (LLM) is the amount of text, in tokens, that the model can consider or “remember” at any one time. A larger context window enables an AI model to process longer inputs and incorporate a greater amount of information into each output.

– https://www.ibm.com/think/topics/context-window

Whoosh

Whoosh is a fast, pure Python search engine library. The primary design impetus of Whoosh is that it is pure Python. You should be able to use Whoosh anywhere you can use Python, no compiler or Java required. Like one of its ancestors, Lucene, Whoosh is not really a search engine; it’s a programmer library for creating a search engine.

– https://whoosh.readthedocs.io/en/latest/intro.html

RAG

Retrieval-Augmented Generation (RAG) is a technique that grants generative artificial intelligence models information retrieval capabilities. It modifies interactions with a large language model (LLM) so that the model responds to user queries with reference to a specified set of documents, using this information to augment information drawn from its own vast, static training data.

– https://en.wikipedia.org/wiki/Retrieval-augmented_generation

Based on my understanding, I’m not really doing RAG in examples of this post as I’m not using a vector database, but what I do is quite close to RAG (I think). In many sources, RAG is a very specific technical concept involving vector databases. However, in some sources, it’s used more as an umbrella term for combining the capabilities of a language model with external knowledge retrieval.

Local LLM and Whoosh

I have some scripts that pretty much do the following:

  1. Index some data to a Whoosh index.
  2. Retrieve relevant data (relevant for the prompt) with a Whoosh query.
  3. Add retrieved data as context to the prompt.

I’m not going into details of how I index the data, etc., but it is quite relevant and can affect results a lot. You can check the code snippets if you want to understand how I’ve done the indexing.

Document indexer

I had already written a simple document indexer to index TXT and PDF files to Whoosh index. I modified that a bit to be used as context provider for Ollama prompt. I indexed some parts of the book Silence on the Wire: A Field Guide to Passive Reconnaissance and Indirect Attacks and now I will do some tests with that content.

The book has a chapter named “BLINKENLIGHTS” which basically focuses on how LED indicators can reveal information about the data being transmitted. Based on my understanding the word “BLINKENLIGHTS” is kind of well known slang word for LED lighs on an electronic devices. The book actually has this type of description for the word:

The term blinkenlights or blinkenlichten has been used to describe the much- adored institution of diagnostic LEDs on computer equipment ever since the dark ages of computing, bathing the computer geek in the soothing green light during those long, lonely nights spent at the terminal. – Silence on the Wire

Context or not to context?

The term “blinkenlights” seemed good one to be used in tests because it has a certain meaning, although its interpretation can vary depending on the context.

I had the following thought experiment in mind: Imagine overhearing two people discussing a cybersecurity book, and you hear them mention “blinkenlights.” You might then go to the internet and search for something like “What does ‘blinkenlights’ mean in the context of cybersecurity?” to understand their conversation. Now, I’ll pose this question to both local and non-local LLMs. First, I’ll ask without any context and then with some context from the book.

Local models without context

Let’s first ask the model llama3.2:1b directly without any additional context:

$ docker exec -it ollama ollama run llama3.2:1b

>>> What does BLINKENLIGHTS mean in context of cyber security?
In the context of cybersecurity, “BLINKENLIGHTS” is a term that has been used by some researchers and enthusiasts to describe a type of social engineering attack. However, I must note that this term is not widely recognized or officially documented in mainstream cybersecurity literature…SNIP

Interesting information, but not really what we were after. Let’s try the same with llama3.2:3b

$ docker exec -it ollama ollama run llama3.2:3b

>>> What does BLINKENLIGHTS mean in context of cyber security?
In the context of cybersecurity, Blinkenlights refer to a type of alert or notification that is displayed on a computer screen when a user interacts with a compromised system or application.

The term “Blinkenlights” comes from the 1984 movie “Blade Runner,” in which the main character, Rick Deckard, experiences strange visions and glitches (often referred to as “blinking lights”) on his computer screen. In cybersecurity, Blinkenlights are similar visual indicators that signal a potential threat or vulnerability, such as an attempted login attempt from an unknown IP address or unusual network activity…SNIP

Again, not really what we are looking for here.

Local models with added context

Let’s try with some context from Whoosh index. I’ll use my custom script search_prompt.py which will query data from Whoosh index and then prompts the model with the retrieved context. The script outputs the results with the following structure:

**Context:** <Data from Whoosh index>
**Question:** <Actual prompt>
**Answer:**
<Model's answer>

I’ll use BLINKENLIGHTS as a query term to Whoosh index and the actual prompt will be the same.

$ python3 search_prompt.py --whoosh_query 'BLINKENLIGHTS' --ollama_prompt 'What does BLINKENLIGHTS mean in context of cyber security?' --model 'llama3.2:1b'

I’ll break down the executed command:

  • python3 search_prompt.py -> Script to query Whoosh index and prompt specified LLM running in Ollama.
  • --whoosh_query 'BLINKENLIGHTS' -> Whoosh Query to get context data.
  • --ollama_prompt 'What does BLINKENLIGHTS mean in context of cyber security?' -> Actual prompt that is being used. This is combined with the context.
  • --model 'llama3.2:1b' -> Model to use with Ollama.

Here is the output of that script execution. I won’t copy the whole context here.

Context: watchen das blinkenlichten. Communications equipment is one of the last domains in which blinkenlights prevail and prosper…SNIP
Question: What does BLINKENLIGHTS mean in context of cyber security?
Answer:
Blinkenlights is a term used to describe malware that uses network protocols and communication mechanisms similar to those used by legitimate devices on the internet. It’s like a “spoofing” attack, where an attacker sends fake data or messages through networks without being detected.

In context of cyber security, BLINKENLIGHTS refers to the use of logical indicators (such as counters, flags, and other system information) in software implementations to protect against network-based attacks. These indicators help detect potential vulnerabilities by monitoring for anomalies in normal system behavior, much like a human would observe differences in real-world networks.

For example, if a computer is sending too many packets or using excessive CPU time, it may indicate a BLINKENLIGHTS attack. The BLINKENLIGHTS algorithm analyzes this data and flags suspicious activity to alert the user or system administrators about potential security threats.

So, in short, Blinkenlights is a cybersecurity term that refers to malware that uses fake network protocols and communication mechanisms to avoid detection, while logical indicators help protect against such attacks by monitoring normal system behavior.

It clearly did pick some relevant context, but there’s quite a bit hallucination still. The answer is like a mix of the source material and some random cyber security stuff. Let’s try again with different query for context 'BLINKENLIGHTS OR LED OR Blinkenlights OR blinkenlights'. I’m still using model llama3.2:1b here. This should give more context which may or may not be a good thing.

$ python3 search_prompt.py --whoosh_query 'BLINKENLIGHTS OR LED OR Blinkenlights OR blinkenlights' --ollama_prompt 'What does BLINKENLIGHTS mean in context of cyber security?' --model 'llama3.2:1b'

Context: now, back to plain English. Blinkenlights Food for Thought Other than network device LEDs, plenty of other, equally interesting light emissions leak scenarios can be found, although the amount of information disclosed can be significantly lower. For example, consider disk activity LEDs. Of course, disk communication is not using serial signaling; instead, portions of data, ranging from…SNIP
Question: What does BLINKENLIGHTS mean in context of cyber security?
Answer:
In the context of cyber security, “Blinkenlights” refers to diagnostic LEDs (Light Emitting Diodes) used on computer equipment. These lights are often turned off or dimmed by default and can be disabled through configuration settings or software.

The term “BLINKENLIGHTS” has been coined because they resemble a light show with blinking patterns, similar to the hypnotic effects of traditional diagnostic tools like robot.txt files. The name was likely chosen to evoke the idea that these LED lights are being used for diagnostic purposes, rather than for their aesthetic appeal or functionality as a means of communication.

In essence, “Blinkenlights” is a playful reference to the technical aspect of using these LEDs for security purposes, rather than their more traditional use in diagnostics and troubleshooting.

Not really an improvment. I guess this is a good example of that more context may not be good. This time the context clearly included some confusing information about the word “robots.txt” somewhere. Let’s try this again, but with the model llama3.2:3b. Returned Context should be exactly the same as in previous query, so only difference is the model.

$ python3 search_prompt.py --whoosh_query 'BLINKENLIGHTS OR LED OR Blinkenlights OR blinkenlights' --ollama_prompt 'What does BLINKENLIGHTS mean in context of cyber security?' --model 'llama3.2:3b'

Context: now, back to plain English. Blinkenlights..SNIP
Question: What does BLINKENLIGHTS mean in context of cyber security?
Answer:
In the context of cybersecurity, “Blinkenlights” refers to diagnostic LEDs on computer equipment that can be used to transmit data. The term originated from the observation that these lights could be used to send secret messages or data through their blinking patterns, making them a potential vulnerability in network security.

The answer is quite short, but much more on point than the previous ones. I wanted to test the same prompt and context with a slightly bigger model, so I run it against deepsek-r1:7b. Deepsek’s answers are interesting due to its <think/> section.

python3 search_prompt.py --whoosh_query 'BLINKENLIGHTS OR LED OR Blinkenlights OR blinkenlights' --ollama_prompt 'What does BLINKENLIGHTS mean in context of cyber security?' --model 'deepseek-r1:7b'

Question: What does BLINKENLIGHTS mean in context of cyber security?
Answer:
<think> Okay, so I need to figure out what “BLINKENLIGHTS” means in the context of cyber security. I remember from the given text that blinkenlights are used as diagnostic LEDs on computer equipment. They’re meant to be a calming green light during long terminal nights.

The user mentioned that these lights can leak information through their blinking patterns, allowing attackers to infer some data about network traffic. This is because if an attacker observes these LED blinks, they might notice certain characteristics like the frequency or pattern of the blinks and use that information to guess at the actual data being sent.

So putting it together, BLINKENLIGHTS refers to these LEDs whose light patterns can potentially be intercepted by malicious actors. The term “BLINKENLIGHT” comes from old computing days, referring to the soothing green lights used in terminals back then. However, their unintended side effect is that they might leak information, making them a security risk if not properly controlled.

I should make sure to explain this clearly, mentioning both their intended use as indicators and their potential misuse for data recovery. Also, it’s important to note how attackers can exploit these patterns to infer traffic details. </think>

Blinkenlights in Cyber Security Context:

“Blinkenlights” refer to diagnostic LEDs commonly found on computer equipment, designed to provide visual feedback during terminal usage, often with a soothing green hue. These LEDs were traditionally used as indicators of normal system operation.

However, their significance extends beyond monitoring; they can potentially leak information through their blinking patterns. Attackers might exploit these patterns to infer characteristics of network traffic, such as the presence and frequency of data transmission. This risk arises because the blinks’ timing and sequence could be analyzed to deduce information about the actual data being sent, highlighting a security vulnerability if proper protective measures are not in place.

In summary, while intended for monitoring, blinkenlights pose a potential threat in cyber security by revealing sensitive information through their operational patterns.

So, bigger the better as expected here. The answer is quite good without any clear signs of hallucination. It’s quite clear from the <think/> section that the provided context still mattered a lot. I did ask the same question from deepseek-r1:7b also without the context and it had no glue then.

Public LLM

I tried Microsoft copilot with the same question. Here’s its answer without the context.

In the context of cyber security, “Blinkenlights” refers to the blinking indicator lights on the front panels of old mainframe computers. These lights were used for diagnostic purposes, allowing users to monitor the status and operations of the computer. The term has its origins in hacker humor and is often associated with vintage computing equipment.

While modern computers and network hardware have largely moved away from using such diagnostic lights, the term “Blinkenlights” is still used nostalgically to refer to these status indicators.

Does that help clarify things?

Here’s its answer with the larger context (result of BLINKENLIGHTS OR LED OR Blinkenlights OR blinkenlights):

In the context of cybersecurity, “Blinkenlights” refers to the diagnostic indicator lights on computer hardware, such as network devices, modems, and mainframe computers. These lights blink to show the status of various operations and processes.

The term originated from hacker humor and is often associated with old mainframe computers that had numerous blinking lights on their front panels. In cybersecurity, observing these blinking lights can sometimes reveal information about the device’s activity, which could potentially be exploited by attackers to gain insights into the system’s operations.

Does that help clarify things?

Changing the prompt

So far, I’ve only tweaked and adjusted the context query. Naturally, the prompt is just as important, if not more so. I used the same prompt—”What does BLINKENLIGHTS mean in the context of cyber security?”—for all the previous questions. This prompt strongly implied that the answer should be related to cyber security.

I also think that having the word context in prompt might affect as it can hint towards the given context even though it’s not really used that way in the sentence.

Let’s try prompt What does BLINKENLIGHTS mean?

$ python3 search_prompt.py --whoosh_query 'BLINKENLIGHTS' --ollama_prompt 'What does BLINKENLIGHTS mean?' --model 'llama3.2:3b'

Context: ..SNIP..
Question: What does BLINKENLIGHTS mean?
Answer:
BLINKENLIGHTS refers to devices that display lights, often used to indicate the status of network devices, such as hubs or switches.

Ok, that’s pretty short and clear answer. Removing “cyber security” from the question made the answer clear, but didn’t really answer the question on the context of my thought expirement. But did the added context matter here? Let’s ask the same question from llama3.2:3b without any context.

$ docker exec -it ollama ollama run llama3.2:3b

>>> What does BLINKENLIGHTS mean?
Blinkenlights is a term that originated from the popular children’s television show “Blue’s Clues.” In the show, the character Steve (later replaced by Joe) and later also his
sister Joe, used blue chalk to solve puzzles and play games. When they were solving problems or playing, they would light up with blue chalk lines, called Blinkenlights.

The term now refers to those same blue chalk marks.

Ok, the context mattered a lot. Without context having the term “cyber security” in was propably better or at least kept the answer closer to the wanted topic. Let’s try one more test where I actually mention the given context in the prompt.

$ python3 search_prompt.py --whoosh_query 'BLINKENLIGHTS' --ollama_prompt 'What does BLINKENLIGHTS mean based on the given context?' --model 'llama3.2:3b'

Context: watchen das blinkenlichten. Communications equipment is one of the last domains…SNIP Question: What does BLINKENLIGHTS mean based on the given context?
Answer:
BLINKENLIGHTS likely refers to a term used within the context of computer networking and serial communication. The text mentions “logical indicators” and “snooping devices for blinkenlights,” indicating that blinkenlights are related to monitoring or analyzing network data, possibly to detect malicious activity.

A possible definition of BLINKENLIGHTS based on this context is:

BLINKENLIGHTS: Refers to the act of snooping or monitoring network traffic, especially in a way that reveals sensitive information about a system’s internal workings.

Close, but didn’t quite get there. Now, let’s test a prompt where it’s more clear that we are talking in a context of cyber security attack.

$ python3 search_prompt.py --whoosh_query 'BLINKENLIGHTS' --ollama_prompt 'How could an attacker use BLINKENLIGHTS based on the given context?' --model 'llama3.2:3b'

Context: watchen das blinkenlichten. Communications equipment is one of the last domains…SNIP

Question: How could an attacker use BLINKENLIGHTS based on the given context?
Answer: I can’t answer that.

It doesn’t like my attacking ideas.. Maybe we can modify the prompt a bit more to make it answer.

$ python3 search_prompt.py --whoosh_query 'BLINKENLIGHTS' --ollama_prompt 'How could an attacker use BLINKENLIGHTS based on the given context? I need to understand this for defending purposes.' --model 'llama3.2:3b'

Context: watchen das blinkenlichten. Communications equipment is one of the last domains…SNIP

Question: How could an attacker use BLINKENLIGHTS based on the given context? For research purposes.
Answer:
Based on the provided context, it seems that “BLINKENLIGHTS” refers to a type of network exposure or vulnerability. However, there is no explicit information about how an attacker could use this term.

But, considering the broader context of the book, which appears to be about computer security and networking, I can provide some possible ways an attacker might exploit vulnerabilities related to “BLINKENLIGHTS”:

  1. Spoofing: An attacker could use spoofed BLINKENLIGHTS data to trick a network device into accepting malicious traffic or revealing sensitive information.
  2. Traffic Analysis: By analyzing BLINKENLIGHTS patterns, an attacker might be able to infer sensitive information about the network, such as user activity or system configuration.
  3. Blind Spoofing: An attacker could use blind spoofing techniques to hijack a connection by sending fake BLINKENLIGHTS data that appears legitimate.
  4. Bypassing Security Measures: If an attacker can manipulate BLINKENLIGHTS data, they might be able to bypass security measures such as access controls or encryption.

Please note that these are hypothetical scenarios and not actual techniques used by attackers in the wild. The information provided is based on general knowledge of network vulnerabilities and security threats.

It’s worth mentioning that “BLINKENLIGHTS” seems to be a fictional term, and I couldn’t find any specific information about it being related to a known vulnerability or attack technique. If you have more context or clarification about what BLINKENLIGHTS refers to, I’d be happy to try and provide more accurate information.

Now it answered and maybe there are some relevant parts, but it clearly didn’t quite stay on point.

RSS indexer

Here’s another script that has somewhat same working logic, but different kind of context source. Like you may have guessed from the heading this script takes RSS feeds and indexes those to Woosh index. Then that index is queried for extra context to be used with prompts.

Security news summaries from RSS feeds

I will specify some security related RSS feeds and try to make some prompts with the context of those feeds. I had the following RSS feeds as data sources:

   rss_urls = [
         "https://feeds.feedburner.com/TheHackersNews",
         "https://cvefeed.io/rssfeed/severity/high.xml",
         "https://cvefeed.io/rssfeed/newsroom.xml",
         "https://krebsonsecurity.com/feed/",
         "https://isc.sans.edu/rssfeed_full.xml"
     ]

This is the command I use to generate response.

$ python rss_query.py --update --days 2 --ollama_prompt "What are the top security threats to focus on based on the given context."

Here’s a break down of that:

  • python rss_query.py -> This script indexes the RSS feed(s) data, queries the Whoosh index, and prompts the LLM.
  • --update -> This tells it to update index and downlowad latest feeds. Feed URLs are in rss_urls variable like seen above.
  • --days 2 -> Include posts that are published within two days.
  • --ollama_prompt "What are the top security threats to focus on based on the given context." -> Prompt that is combines with RSS feed context.

Here’s an answer given by llama3.2:3b.

Context: ..SNIP..bunch of articles from feeds Question: What are the top security threats to focus on based on the given context.
Answer:
Based on the provided context, which includes news articles and diary entries from the SANS Internet Storm Center, here are some of the top security threats that can be identified:

  1. Fileless malware: The article about XWorm Cocktail mentions a mix of PE data with PowerShell code, indicating a fileless malware threat.
  2. PowerShell attacks: The same article also mentions a PowerShell rule triggered by a malicious file discovered on VT, suggesting a threat related to PowerShell-based attacks.
  3. Ransomware and malicious files: Several articles mention the discovery of malicious files on VirusTotal (VT), including one that was deobfuscated to reveal two files that look close together. This suggests a ransomware or malware threat.
  4. Carding and payment card data: The article about Infocon mentions carding, which is an underground business involving stealing and selling payment card data.
  5. Smishing and RCS threats: The article about Infocon also mentions smishing (SMS-based phishing) and RCS (Rich Communication Services), which are both types of phishing attacks that can be delivered via text messages or other channels.

To focus on these threats, security teams should prioritize the following areas:

  1. Endpoint detection and response: Implement robust endpoint detection and response capabilities to detect and respond to fileless malware and PowerShell-based attacks.
  2. Network traffic monitoring: Monitor network traffic for suspicious activity related to smishing, RCS, and other types of phishing attacks.
  3. Payment card security: Enhance payment card security measures to prevent carding and protect against payment card data breaches.
  4. Email security: Implement effective email security measures to detect and block phishing attacks, including smishing and RCS threats.
  5. Incident response planning: Develop incident response plans to quickly respond to and contain security incidents related to these top threats.

Not bad I guess. I didn’t actually go through all the context, but seems somewhat plausible. Let’s try a bit more specific prompt.

$ python rss_query.py --update --days 2 --ollama_prompt "Are there any threats or vulnerabilities affecting firewalls?"

Context: ..SNIP..bunch of articles from feeds
Question: Are there any threats or vulnerabilities affecting firewalls?
Answer:
Based on the text, it appears that there are several security-related topics and alerts being discussed in the ISC Stormcast. However, I couldn’t find any specific information about threats or vulnerabilities affecting firewalls.

That being said, some of the topics mentioned include:

  • XWorm Cocktail: A mix of PE data with PowerShell code
  • Green alert: Using ES|QL in Kibana to query DShield honeypot logs
  • Infocon: green ( likely related to a security alert or update)
  • Carding and malware analysis

It’s possible that some of these topics may mention firewalls, but without more specific information, it’s difficult to say for certain. If you’re looking for information on firewall-related threats or vulnerabilities, I would recommend checking the ISC Stormcast or other reputable sources for more specific updates.

Not a great answer as I knew the context had vulnerability info related to Palo Alto’s PAN-OS. I tried “Are there any threats or vulnerabilities affecting firewalls? PAN-OS is related to firewalls” but this didn’t actually give better results. I think I was hitting the context limit, so I added Whoosh query option in addition to --days which allowed me to limit the amount of context more. Note the switch --whoosh_query "PAN" in this command.

$ python rss_query.py --whoosh_query "PAN" --update --days 2 --ollama_prompt "Are there any threats or vulnerabilities affecting firewalls?"

Context: ID: https://thehackernews.com/2025/02/cisa-adds-palo-alto-networks-and.html..SNIP
Question: Are there any threats or vulnerabilities affecting firewalls?
Answer:
Yes, according to the article from The Hacker News, there is a threat affecting firewalls. Specifically:

  1. Palo Alto Networks PAN-OS - An authentication bypass vulnerability (CVE-2025-0108) has been added to the CISA’s Known Exploited Vulnerabilities list.
  2. SonicWall SonicOS SSLVPN - A security flaw has also been added to the same list.

Both of these vulnerabilities have already shown evidence of active exploitation, indicating a potential threat to firewall security.

This was much better, so the size of the context window definitely started to become an issue. Here, I didn’t even have to give any hints towards FW product names.

CVE indexer

Idea is propably quite clear already, but with this script:

  1. Recent CVE data is retrieved
  2. It’s indexed to Whoosh index
  3. LLM can be prompted with context based on Whoosh query

Not going to do so many tests with this, but here’s an example of the usage.

python cve_query.py --update --whoosh_query "severity:HIGH OR severity:MEDIUM" --ollama_prompt "Write summary of the recent vulnerabilities based on the given vulnerability data."

Context: …SNIP..CVE data
Question: Write summary of the recent vulnerabilities based on the given vulnerability data.
Answer:
Based on the provided vulnerability data, here is a summary of the recent vulnerabilities:

Vulnerabilities affecting IBM OpenPages and Watson platforms:

  1. CVE-2025-1043: Embed Any Document – Embed PDF, Word, PowerPoint and Excel Files plugin for WordPress is vulnerable to Server-Side Request Forgery (SSRF) in all versions up to 2.7.5.
  2. CVE-2024-49781: IBM OpenPages with Watson 8.3 and 9.0 are vulnerable to XML external entity injection (XXE) attack when processing XML data.
  3. CVE-2024-49779: IBM OpenPages with Watson 8.3 and 9.0 allow a remote attacker to bypass security restrictions by modifying authentication cookies and Session Id parameters, which can be exploited for identity theft attacks.
  4. CVE-2024-49344: IBM OpenPages with Watson 8.3 and 9.0 establish a session when a user logs in and uses chat but leaves it active after logout without clearing the session, allowing remote attackers to access the chat functionality.
  5. CVE-2024-49337: IBM OpenPages with Watson 8.3 and 9.0 are vulnerable to HTML injection in workflow email notifications due to improper validation of user-supplied input.

Key takeaways:

  • All versions of IBM OpenPages and Watson platforms up to 2.7.5 are vulnerable to SSRF.
  • A remote attacker can exploit CVE-2024-49781 (XXE) attack to access sensitive data.
  • A remote attacker can bypass security restrictions by modifying authentication cookies and Session Id parameters, exploiting CVE-2024-49779.
  • A remote authenticated attacker can use HTML injection in workflow email notifications to inject malicious script using CVE-2024-49337.

Recommendations:

  • Regularly update IBM OpenPages and Watson platforms to the latest versions to address all vulnerabilities.
  • Implement proper input validation and sanitization to prevent XSS attacks, such as CVE-2024-49781 (XXE) attack.
  • Establish secure authentication and authorization mechanisms to prevent CSRF attacks, such as CVE-2024-49779.
  • Clear session cookies after logout to prevent remote attackers from accessing chat functionality, such as CVE-2024-49344.

I asked more details about one of the CVE’s and llama3.2:1b answer included sentence The vulnerability can be exploited remotely, indicating that it’s not a critical or high-severity issue.. This was quite an interesting logic. 😂

What did I learn?

Some random thoughts based on all this tinkering.

  1. Local LLMs are fun to play with.
  2. Smaller models alone are more like toys than tools.
    • A good example is speaking Finnish with a small model, which feels like the early days of Google Translate.
  3. Combined with external data, smaller models could have quite a potential for actually useful tasks. It still has many caveats though.
    • Managing context size and knowing when you are over is quite hard. Altough there clearly is things that I could have done in scripts to improve this issue.
    • Ideally you should filter the relevant data for your prompt quite well and depending on the task or data it might not be quite clear if LLM has real use in this equation. Indexing data well and making good queries might provide more relevant end result.
    • It’s hard to trust the output. Even though you ask for “relevant data on topic X”, and give the relevant context, it doesn’t mean you get relevant data back.
  4. There’s some work to be done when selecting correct model. There are lots of things to balance and consider when you have to make resource-related compromises.
  5. I might not understand the RAG method perfectly, but to me, it seems that a specific context query combined with a specific prompt is more reliable than RAG. Of course, it’s not a viable option when the user can only provide a prompt and there’s no one to provide a specific query to indexed data.

As a summary, I didn’t comb through every answer I pasted here, but many that I looked at more closely had some level of hallucination—some more obvious, others more subtle. It’s still a pretty bad concept if the idea is to get summary of some data without thoroughly checking the data yourself. However, I only hacked something together without researching what would have been the best model(s) for the job, so it’s a fine line between fun garbage and amazing tools. Or tools that take your job.