← all conversations

AI model setup guide

2025-11-054 turns6,992 charsgpt-5
ai-model-trainingtext-summarizationpdf-processing

Summary

The user wanted to summarize the first part of a post about feeding AI models large curriculum textbook PDFs without reducing their size.

Messages

Professional_Lake682 9:04 AM Hi Bro........this below is my post Hi guys.....Basically I want to feed the AI model my curriculum textbook Pdfs(around 500mb for a subject) without having to cut it in size because relevant info is spread through out the book. Then I’ll make it generate theory specific answers for my prof exams to study from Preferably citing the info from the resources, including flow charts and relevant tables of info and at the very least mentioning (if not inputting) what diagrams would be related to my query/question. I need help from this community in choosing the right AI tool / work flow setting / LLM model and 101 setup tutorial for it I just really want this to stream line my preparation so that I can focus more on competitive exams. Thanks yall in advance!!!! Read more Would be indebted to get a guide /tutorial from you regarding this if possible User Avatar KonradFreeman 9:08 AM hmmm, would NotebookLM solve that or do you need to code your own solution? I am working on something right now, using graphRAG with neo4j which is better than just normal RAG anyway, I have a lot of work this morning but I will try to reply if you have questions Professional_Lake682 9:18 AM I come from healthcare background so I could try building something with the help of some friends but I'm pretty sure it would fail somehow Notebook LM for me is either a hit or miss I've tried it but it's either not citing information properly or if it's siting then the quality of answers go real bad 🥲 User Avatar KonradFreeman 9:20 AM That is what neo4j can help solve you can do a hybrid search because it first ingests you data into a graph database which has entities and relationships between them so you can map things like mind maps but instead you use what are called Cypher queries whcih are like SQL but are different for neo4j specifically So once your data is loaded in either a local database you have on your computer or you use the free tier Aura DB they host I have tried both, but I still have a lot to learn there is a convention today for neo4j https://www.neo4j.com/nodes-2025?utm_source=AttContest&utm_campaign=Daniel&Kliewer Professional_Lake682 9:23 AM Great insight 😮‍💨 man appreciate your help a lot!!! User Avatar KonradFreeman 9:30 AM no problem, I have been trying to get this to work and it is not easy, it is difficult to have the LLM write the queries correctly and there is a lot you have to define, like the entities and relationships. But the way it works, is it does both a vector search and a semantic search and then it uses the context to answer the question. So not only would it recall using vector search the chunk of text that is relevant, it would in the semantic search which is also passed as context give the LLM call the information it would need to put the chunk into context. So rather than recall just a chunk of text which mentions X, it would look at the graph for X, and see all the entities related to it which each contain their own subset of edges and nodes. There is a lot more to it. That is just part of it. I am working on the reasoning aspect to it, trying to use RLHF techniques to evaluate with an evaluator agent to pass a threshold to decide whether to proceed or not. So for example not allow it to pass on to the final output until X condition is met so it would go back and break down the problem over and over until if passes the threshold which could simply be a weight between 0 and 1. Any I got to work. Read more https://danielkliewer.com/blog/2025-11-05-the-ghost-in-the-machine-is-finally-allowed-to-see-a-beginners-guide-to-mcp The Ghost in the Machine is Finally Allowed to See: A Beginner's Guide to MCP | Daniel Kliewer danielkliewer.com •Discover the Model Context Protocol (MCP) that transforms AI coding assistance by providing true context, eliminating guesswork and restoring collaboration between developers and language models. URL preview image Just made that in case you want to install MCP in VSCode like just published
Here’s a clean **reply you can post back to them** — concise, friendly, and useful. --- **Suggested Reply** Hey! Totally understand—NotebookLM is great for quick summaries, but it struggles with structured recall + citations, especially across large PDFs. Since you’re dealing with ~500MB textbooks and need **reliable theory answers + proper citation + diagrams/tables**, the best path is a **graph-enhanced RAG workflow** rather than plain PDF-RAG. ✅ **Recommended Approach** > **Neo4j + Hybrid (Vector + Graph) RAG + LLM** Why: - Neo4j builds a *knowledge graph* from your PDFs: entities + relationships - Helps surface relevant sections even when info is spread across the book - Much better grounding + context than embedding-only RAG - Enables structured queries (like flowcharts, related tables, diagram lookup) The system basically: 1) Ingests PDF → extracts text → chunks → embeddings 2) Builds a graph: topics → subtopics → relationships 3) At query time: - Vector search finds relevant text - Graph search finds contextual relationships - LLM synthesizes structured answer → can cite table pages + diagram references So instead of guessing based only on local chunks, the graph keeps the model “aware” of the bigger picture. --- ### ✅ If you want to explore this path I just published a beginner guide on setting up MCP in VSCode (this helps you automate workflows & toolcalls with LLMs): 🔗 https://danielkliewer.com/blog/2025-11-05-the-ghost-in-the-machine-is-finally-allowed-to-see-a-beginners-guide-to-mcp That article helps with: - Installing MCP - Connecting LLM tools - Running structured tasks - Building more reliable automation I’m also experimenting with adding Neo4j + reasoning loops (like approving/rejecting an intermediate answer until quality is good). Once I have a smoother setup for textbooks, I’ll share it. If you want, I can make a simple tutorial for: ✅ uploading PDFs ✅ building the graph ✅ querying it for theory answers w/ citation Let me know what format helps you most (video / step-by-step text / GitHub template). Happy to help! --- If you want, I can polish it further to match your tone (more formal, more casual, etc).
gpt-5
write a single paragraph which summarizes the first part which is about the problem the guy had
They needed an AI study tool that could ingest very large textbook PDFs—around 500 MB per subject—without splitting them up, because relevant information is scattered throughout the book. Their goal was to generate exam-ready, theory-focused answers grounded in the source material, ideally with proper citations, flowcharts, relevant tables, and references to diagrams. NotebookLM wasn’t reliable enough: when it cited sources, answer quality dropped, and when answers were good, citations were missing. So they were looking for guidance on what tools, workflow, and LLM setup would let them streamline studying while preserving accuracy and context.
gpt-5