From 012e111a72d39f3f311960a401ce416f93758f4f Mon Sep 17 00:00:00 2001 From: Lore Engine Dev Date: Thu, 18 Jun 2026 23:45:59 -0400 Subject: [PATCH] docs: README + plan/ADR for slice 5 (Neo4j GraphBackend) Updates the POC README with a 'Storage backends' section documenting the GraphBackend Protocol, the two implementations (InMemoryGraph and Neo4jGraph), and the LORE_GRAPH_BACKEND env-var selection in the MCP entry scripts. Adds the slice 5 plan doc (docs/plan/05-slice-neo4j-backend.md in the design repo) and ADR 0011 capturing the Protocol + dual-write model decisions. --- README.md | 55 +++++++++++++++++++++++++++++++++++++ lore_engine_poc/.graph.pkl | Bin 165970 -> 165970 bytes 2 files changed, 55 insertions(+) diff --git a/README.md b/README.md index ce8193d..db3b2c0 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,61 @@ The 44 other MCP tools, the consistency engine, the TypeTemplate polymorphic extension, the plane model, the MCP server wiring. All deferred to follow-up slices per the design. +## Storage backends (slice 5) + +The 36 MCP tools and the consistency engine read from a +`GraphBackend` Protocol (`lore_engine_poc/graph_backend.py`) +with two implementations: + +* **InMemoryGraph** (default, pickle-backed). The in-memory + dict-of-dicts from slices 1-11. Used for fast tests, the + CI test suite, and the Docker image's baked-in + `.graph.pkl` (slice 11). +* **Neo4jGraph** (slice 5.3+). A `neo4j:5` container with + the reified `:Relation` shape (ADR 0009). The production + graph substrate per ADR 0008. + +The MCP entry scripts (`scripts/05_mcp_server.py`, +`scripts/06_mcp_http_server.py`) select the backend at +startup via `LORE_GRAPH_BACKEND`: + +```bash +# Pickle (default) — load .graph.pkl +python3 scripts/05_mcp_server.py +# or +python3 scripts/06_mcp_http_server.py + +# Neo4j — connect to bolt://$LORE_NEO4J_URI +LORE_GRAPH_BACKEND=neo4j LORE_NEO4J_URI=bolt://localhost:7687 \ + python3 scripts/06_mcp_http_server.py +``` + +Compose brings up the Neo4j stack with a profile: + +```bash +# Pickle-backed MCP (slice 11 default) +docker compose --profile pickle up -d + +# Neo4j-backed MCP (production substrate, slice 5) +docker compose --profile neo4j up -d +``` + +The Neo4j compose stack includes: + +* `neo4j:5` — the database (`NEO4J_AUTH=none` for loopback; + **switch to a username/password before any non-loopback + exposure**). +* `lore-engine-ingest` — one-shot job that runs + `01_ingest.py --write-neo4j --skip-cognee` after Neo4j + is healthy. +* `lore-engine-mcp-neo4j` — the MCP HTTP server reading + from Neo4j (depends on both `neo4j` healthy and + `lore-engine-ingest` exited 0). + +The pickle profile keeps working exactly as in slice 11.4 +(pickup unchanged, MCP server on port 8765, baked +`.graph.pkl`). + ## Run ```bash diff --git a/lore_engine_poc/.graph.pkl b/lore_engine_poc/.graph.pkl index a9020a890b569e9f38992c349e6dd91fe69c62d1..d22f04e601a9ae6ea16f7f2a858aa37fbf7d957a 100644 GIT binary patch delta 7550 zcmcgwd3aP+miN9;$;$>IArXip1tbubP_L?9Z6Gw81{%VWKtLAdZI!A=B^4?OTLZB{ zRtpF@rI#S9fCvHx3N04*Zd^f;rPa1uHXB(6MYL5Ebe_gIza@I({gMVH+xYF2NwED0&ELzWzxM}+XdO@){F2ro*qYL~s9 z=#=Cv!fQ^KDma`jSY0p`HZSc6pQS9~T^`x3*&S|JWo2hl;9P10Z*{v=uc%s~b?b6C zp9Z|*al1vU*9-H~SMW9)x+7_B{>@ru(fi8j#=1Bb@- ztZKK67gdMT=1?@aI~#br$Duj}xBRmubI!xOX4NFC>Q!KO&ZE3Xu{oU>(HOV2M5eYiwZs47=!!6q| zTb*+6gpYE!^Hzr>Ic2xi5WjlYI$jec!6pbEtie@YwW_kyrMY0i-CKD#!SVQr@+yi2l5Ub|p}Zsx`$6%*uDoQAG~&dYeW=yZB*ssj#oUd2nQTl9LYHW=0A z9PbqEiYj0!U+H=$6n72q0`91a&5Ik$d8bp9Rj*q>Vgl^U8^XI)%^|uJuc5gte;Y4Y z9g5fMabVSDP`fSV9afK~YF0aZ*=;Q^X)eKv6@)3tQ{hH;%43m5>`4VX=QC_0r{-4e z3MBQM05^(K;nN=NF)?0f;gAQlY-{-NJvb8WJiDAttmE}Q?PFx<-qIbv1%T>pg@_k zk#~4xm&a!J7-sKx(Il(KE69oheR?Y}$-N=T?s99Q#}WO}jn_}Ky1b%O(BMb+-Qenb zlj(8@F4YFsexIYL%OPTOG!^E@+l6hZLVkMYgU&>Vl=kD;?mqUd@1ntX(&pDe#4LO9;;oJ z-5xFanL6QMO6&Oz{F zm1M6AU-D-0OnU!PF0-{SC!C@c%Gje;o8*DK(v>_0gLg!;!lBY7cwt>GMRhxjQ=7)M zChcfdFRop`h>!Z-2&ddDRo9qH1GN>^k#NQ^DO6PxtTqpk0$Qaig(Bg$<9wQVf>fz$ zA%A7Ld5ph263&IIRk>Xwx{KU#N4WKnIZ7Z{Rc#&@^2-`3Ng3g*4FtNIM@W9UW(&z} zlU}3+%R^F?Z;rWm^pFTW+emU0eLi=`9r{NJ2;ZU_Uvjv0X+>>$c|Z$wuQr$XDWDtA73%x~ zbwYSjdl)$>Cz&tOsXAXyxOZ+pnkXO^tO!7$oTq&ehXs94{oQB>6y)4v7I6a<8pTBAj3Pf}> zb78EdJr~ZQtvQm@33ZOI#l#F{dF2NOc!)>k_}>GZ&hF z8>|ggN`XlD&gNK){gwV;ns}!}pUD!EQW~tq91Zc; z%)uashtU4mS}9bcg)n6YnM7()P-ST-)G?_!o)jjWxm(Juya zd3Wo{OupR+4GR*gW@l=!y^2D?nOY>=9(Fghr*CZ`ooUw|T>HCP5AjJM42DS>bx0}3 z#9|28obcR~E}UKlAC1rLn^6?fB-b<0(bK5Z-Wh(IeqDMpo*6tt+WG7@8?nVo_6B z+D6ybDEQ3am1ETS3OOVN%p)}Hr;K=LgG1UZ^Q>TKI!2dPriD}q6H|tD#$bow$pEH4 z#Ih67C$kR99E@E!I2Z{2Vl+FVpJT;+n-zCH^S%R;^!zf?4#wQyp&aP7?r_c8BUvxW=eBY|<_DiTg=e(WMJwJf()5o@6}{mnLg zA5)T1<_`w6nqORs=tEfnWql!Y$HZikjd*K<&}MvRc)u*OyS{??+m|Ie)p+5|McBnP z5#P3Os?oQ2*%Y4!lcydd8=>HVD`Y2(RV^^+!7b#JvAVuxY(sfER87l=4^|4`k{^Js zk_cad1r8l3VXVJNUC57ytV?cyycq%v*lL`apa^iWQh=D!2!DT8fQQtT@VA`;2>T57 z<61ZJs&TGa{zQ2qjQ95B5>4=X?<=sc$pVwt8YWJZ_kf~DEpQ_6AT0G2k`Ikzbibiz zus;tD_FgC&!55SCIZ&S}nk}WgFm4S%8DfABQB_0t;8%AeRh@ek*I?&Ts)aX;8uN zda|Q=e*?5%Y=OQjXTtnd1yH_UfHSN1lGhFH=%qE#y}mareqMk@^-n?f=>kk%GY`I< zX@SIszrtYrKNHS_SZj%PiR2S<7G^wQiOzcTSq^^tv;b4qE`=OywGFYcIQ5(G&Fu$^ zR$HQ{8uJJoJz|M=d%P=wXJ-p=duv(r-StZe3}0`7#T)+&vE2e}re*L*r*sIErNHe? zoj76wvY}6O#O4qo|86*Veaj+9+--?&dZLiPQf%zoTbF?Jqy_eE`vA_&&4TsYkHQ$0 zsT#{h#A{F5$Zq2#Yi``wu|tGMr?KXr1i@X2F#V}K$kQ#+7oX}*Ab&0k0p;32?XK?d ziiXwOwE(uS`dQ!Ur)^MLFTg-7J!-4p-5n<2bZQ(*hCzD@$ri)f#yyX5oZ&sFd;dg! zGB(2%hU&MT=?Mq+v3qFfSkPoA$5|TTeA6j%vpGzIHDG@aC|zNJ$M!EGQG>DaVS}+X zYK8aLSm5PI33KCt1HIu2mg?|Vfs$C`8|UR4Blq&BZEB*_A-9H~%Vn{2;>5|2UKWxWX%!e2>A^3dt;lz^vna>A;ip1tqL&N9n8^m|JPT#G6^=A`;Pdw`7Y?!k**K(btC_QQ~!@X)%^pf5-O_{qk~>#KHp&Q1p{N&Hzrs% z*6SazJvI~R->?;%?c*bmG1(h)knyF(KC<+`7$R)N=63xoBi}-(({HgQo7Rna$oPEH zpJsc6rR^QI`Wpi_ks8yE@v)>QFs?D(u;J4fadZ>gGp3p{w)gMs*b5BUNQn(H2Iu*B z`Sq62F#dm8Z2wGXb^k}PnP${#IOei@e}zT-SA_a-k~jWEX6&<(TK}9eZ!%yXZTglv zw)``Lj|^a&=h^Bz1{WFpz+g9npBNlx@C*Zvr8~-2Y|htXY;~9cj|?>4{5kmF8%No? zf01z$eudOl8K&1Wmo78-BZH$1-ed4FLax4_E#GADGJ}H*PBYjRo%q&?gw%~}oykJo z7`4CiCqlO*6Om@DCK*tCqJV0r@vj=cwGaUgA(i_u&iw0cbdKh{rxQA*A9y+;~ zqhI&ovf=R?h0&d7j;7GRv>{^jiE||hw4Nt@X_bYffcujIx-*~4rXwsQFWUIYVnUY_ zu35i)seoQf98VSX@uYz2W8=oV(YJUq zou0oI*J!14l94-RUp#k)@3Bvk$+srj&@QemmXSepj_z+`kbPp$XOeblPqeUW}(H zmS!XGB|!2CPb|ktRuej0`7Z||Zah$b6N5SA!ao^2lS$HJKe@>@j?R3iIl@@pJ){Gn z?<{QATGyWpN};KBadv*}RuOqEfxdg1WW`R7AZtzZ%^q0iNn^<6n74!sCbVz?$>Q`b zwCOszi$0`~%-FS3(m$Eb#mMRFi%Hk`=8UQ20~6Jqc$jU*P%7MZB+6%ZGVY%L7;rma?r3Bi)KSuf0HPVm3v`@0>(CcIZt?J8V z#r9q&H3@M3M7vl@7tThg?q{{ zi8FKb4{3OMG(HN(=~#z!u92f%mLQF;7*=@_yPd%;;%KY;FbinIMvrE3$LW$>&PiV) zTpIoC2W(8kmk+YJf>h(D#&~*<(>1wVFZMW@V*PTs5hnWLeDnr&98aKg*{#B*;mI(L zzFZV{bP}O!mSEnJyKr_wcjcjfv4ps2mw4Z`&&R#>7qBf#^SLrQ=2hgTSh&L2)qL)3 z0xg)@LZ|fPOs(i&9VDBkTF{ZveYhR8-p}k<+e+Z>OU-;#+#!JuxR-kkCR}OWyE2Y@p;e34@=!hJ+& X|3xjkD4Uf?518VP^p2racHw^j=yd`W delta 7631 zcmcgvd3Y36w&zx-)0Ge)B!pE-6A}mrbahwt2I4>v140NSY$71lUDfF>I^Cg@u=s=^ zt6&Qur+t^8Yyu*TvNhTu;x>aTC?br`IHL#->!1-G1zhHxTb(e>%zN+i``$M%fBbUq zIp?1J-ad7rKK(>}di`}?^opV?OG2EGPYG|cI%J#O=Z&A|c)|;A$)UKcLi~G!ned9_ zRb^TA#B+^#gm<}B(e4l&@sX)U!mEPA<#8!$e4+_=qGEF^KC2U=w@-st`53Qyyt30O z3h}9F*@X89ZpGzrsj#|m8f;qL6~0Vg%*#%%2SJr1Md(;*&|zh(6(&_ ze4YWkLzUd336^AgYShsam0X4)9jH&8dpA^QIwl$0fYYWwSeNiXEQq_&6`v6~W_k%Qq$VblSvw zeKxmbRUPrkox2d;=k*FsNfcpTm#0l0k7ySJ6$bS4!iuh|ctLVIy*{*%+ig82*kc#0 z$j$B62|nqzjaMZ_61`p*YWKjU+t%?OtH-V?E>?m|yj!$e1)EQR1>LvsZl}}Z^Qtz? z+B7)5F$>n_E<=S@yC`@i=w)s+Io*Eb?{?23kUB>vkPK!llAO#b^c*imx zt7Q{Jw+G-3>`xIT6;+H|`WcAHrnpr>wu8HW126czR;TQ-!iW9WV1YzI6kKvVIKY5+ zMRW<+M~WWJ?=9PTY-xu>wM*DGSOnoo>)vbEm%9b&9!aLj!x8e|0 znCNK4*0;L_MfK`-?{iX~qBlGfF2xXz)n9Qi|Yos_br@LnnO> zY$h)}ch^qraC{)}4<7i{&n6$%$|+*HdwJo_A?taoOYzy=vIN1QkMmwdwuzD=LR!&9 zUQr!ZAHEoHqi7NDQM@kM<8Z^0Vf%Tv>~?rX#TOrPcQ!FuU2eN9+F{D@M&9amdjzM) z0Y41i#7kB*>(hIGT0`*~-YLjF!R>WG-w~6rGenc*Q3cEo?ziw>kIUh6ND6y<8j90k z(8xo)UGz!%Q}*-74ARl*wqfe+_FLLUYcjy*J4Qor)LrP5PjJd^CuV30WQ-ofJDh^z zQ#=@INj7*!uQiE2S+olHzLab?xn-v)dIi{BvVfOmo7>^=$$BcIV{E*No;rLU7rZg1 zf%joWZ65S;MrqXKmISM6bvKlB0Ml6D6`Rj0~bAR)_;o3+@4#u>-T&q^m%;JbD*Os?qo+L+N z{zK99QU2Osu(x@X9EfSfDXm)hzlydU?^n$eDm@W7Xdb1`gU|2l2*zo*Prf@6u8?c| zwW+(OG@CoI0)1(5jW@ zMTmB|NA}hP!Xe#si5yYFA&e;6Zmb-snRly;ZHm=!MMSRh&rAA|Q5^Q5S7u+J!e0~9 z%&1&TV=x;b+IDm;>QOPuk<~#J9lVF7F1$D+5v z0l1Bh8^v|X()_F*#tD&`cv=%sEo{6m*QC9~40NPpe@i;v9*d^w3Tnb36&)Nl&l?O^ zRh!300v@%Rem#UM0MC6{^s`3Nxvt1BV`~()gj9qXVtFgX>U-0hQN; zBbC)&v^!RnXU3vgMd)HtI2gXw&otKgdI!a{)2K?D$lz{Pj04PzoaP9N!qwHco=ay{ z$MmBoc-ffv*GxQ%sVkA^Q2r^>CbQl6@LcS>@p2$&9u|~CaASO~7;Q5$f{u)<#kR!` z!YUwZ9^TGiFcvkH$u-!j71%yZQ)z9D7jF-wP3cLGZYH^=%u-J<9IAjz6Y}~R?osEV zToK-QyyL;zO4OBJ5~#+@Fd`hrfK4Ug$ZQ#l3!hHtd~f>_c`mzww?yGbXYOH_0xg=y z25PEfe(HZKnqDr?R?S006_Ic(nlVw0C^A}@$U==8h6jCg^C+%IN6m=o)~>Or(~H9q zd{EfC0(&QRZKtKP?lU!aR1E4T^=zj(nIvBtRP?5xJ^FB6>iiS5CRVhx(SZjspOtDT z<{uPoJtiE%-YHXMuRo?;U@e0P-m;y__YDNoLK0}+KP&x8tGka zE(=t~{AaY4OljK@;o53e4fayt%M)!g9CK+d_Xn!xAuSf|44Wr+$<|oDs%q+)G!&{6zj|R(Ttm~6~2Yg z_P)HjXluPA&Bgc_V;Jd4Z}hrp@1r&CUES+j9vA%;O&MJ~7Zcx}W$vFNZ!w6e&?9P{ z40&9w*Uyfy7Mdi7@UCKkQ?SAnaR0QUnt)XARa{Td+R!)am;eHT#RDfgQ)zJO{q*bkjw{fIyPQt!f zd1M=^9sq{OyX1<#`YG}jNzpUV>|=RNJ{+lLiI@(?S_^yw7Vy@724@>A@M^4#W%~H+ zLilKn1$xgpN)G6*vW!1GSAvmG3efEKuz7t6(G@onewf!6{=DO+Gw}!WV8ymOney#;!D~iOV@hxmhS9<6;D9LJ_{^b*#xH^ z6kyO}wa_VA2uoiQARK)X_WK!aJv=@m$(z5bknGk^o?5jFwl-Q|%<@|3UEdEbY&Z>*WJ^3RRzTo-g(W`w@fSE)yjp;%YnQ{3LjtT% zjKdEOBLo@;kyl{WQj_*;+`uwEDgOSG%Lx=bXo000{s3pNtefaW zcr>>ST;13Q${w=B@7+{QVBqrt-h{<)ZMG%8@u~iVY}Q@=amzB;9JIjRtsj%~uyWf+ zuxP6V>YsiLiXXQ?lUl~+)9u?OvRxM!_D+HD&QzHBOg=Hi>z|oG$nSIp>+s@sP`k4? z`A#Ps-MN6gtgkjvWTnB$U)#uGXun_~+tfeXn_U0?yj|fE-7C61(7fYRuKTt>HGOFh~y~jec>hY7hP_B zVhxz^#^Ciwz6gdz7C5@T4A&VDKG=^ut4m==>y*M5?;uASO7w}n1p2|9GhdDLa5QjfZt&8K&jn>5A1qA$2NwD<+246G6AZ6h<>LxWcwyGT6s}M_PTN~uFU$yEKQ zU1k3c`A_-3(hf1Le`WA{Mm(c^%+{|lc$&cx24@(&h2XF9Q|(8#Sm}RD`-v^)YZn-o z!_G! z{+7_N|Nm{V{gTk?^y%&Y9-DDity*29SM59t`;Q3qQIjwE*{jn}F*=)wwcoMTHw+Hf zG5nUnCk$R@u!#Zt+S9&ct1S#(XKkHy9Jen;q^?8Hr*&Xd+qa=efp?@lC|xte6cLEPniO*+83rXKN*A83S5P3Jn$ zdo83Oe(=ODjy_nYGY7c3h%3EwxL0kw&f=A#_?wWM&eLqlMFw=}NqmMLJ~Am*+QcvvSERDfGz& zsO6xO^hl)U6N%6ddJ=mgTtMDTqf;A5hXkhGPw1zsk|D5sw4t9_$a8eFmGq%&mXT~| zY_cRWY~%w2Z8sp9;rv8L2U$(%)wllTSj>gT-apxFl zmJlcXDTuK@H-Tg(3L>PjHC@&%DJfFp0#ZnyDJR+VjiuN}HEg?2Na_{oS+;$(xMf>b zhuezW=Cg^{>PQ;{9Xz;&owJzf8sDON32vcllkTEOzhS%2k2mi;OHeeB(jxi++m79v zRMdw$mXi7OOFqfHpYChdA{SVOa!OAwJF#^c8OYNQX5o<=d&q!&T59dVNW=G<|sw>X|$m)#BlabaPsZnufjPNTsvi zBSi`6ZBk^Umj+>Ht$dGsVuV@@hS{duPolP0Mqv)G?;(~%`%~o6Hgre>^HU;|KU72XQ1{p?2d0QlR<k6qc z&}SE;ru00{MyUTPo$9%GI{S)bZIU_zVXOjDVv~&Qs z9V$-cBvASR6CLSo5&X4-J7SMsX7f9s4?}xi%E-4Zfd~QWM9@xK9X;FKtndve~J0U;%0N5~Xtb{})ES BCNcm3