The Problem Matrix#
Our core problem is this: Users struggle with knowledge management on their digital devices, and the problem is only going to exponentially get worse over the coming years. How can we improve upon it? But before we improve upon it, we have to understand the problem deeper. Why do users struggle with document or knowledge management? Trying to pinpoint the problems as – the hierarchical nature of our filesystems, or the data fragmentation across applications – while true, becomes too simplistic.
Over the course of the project, we realized that we have to identify the problem from different facets or aspects. There are the Felt Problems problems that the users feel and describe through words. But there may be problems which do not get expressed by the users, yet are the Root (Causal) Problems. Furthermore, there are desired features by the users. There are constraints and objectives that can be identified (by the researchers) that should be enforced and optimized, respectively, to achieve the ideal knowledge management system. All of this, and more, is what we are trying to discuss in this chapter on “The Problem Matrix”.
Currently, we’ve put a lot of the identified problems, constraints, goals, desired features etc. as a laundry basket collection (but still fairly organized) in this chapter. To arrive at this collection, we’ve utilized internet forum discussions analysis [1], our experiences contributing to and participating in open-source knowledge management communities [2], our experiences while developing LIST project itself, our literature survey done over the years [AN20, BK19, DSG17, DJ20, Gee15, GHBeaudouinLafon20, Gru16, Ken00, Kri80, LCPF26, LM15, Mas21, Mas25, Mil17, SchroderJD22, SM09, SN07, SRvanBarlingenW21, TRB+13, ULG+21, Voi12, VAS09, ZInternationalFfIProcessing72, ArpaciDusseauArpaciDusseau23, JoepMeindertsma, MFiles19, Nayuki17], and, finally, some insights we gained from Philosophy [Kri80].
The Dual Problem#
Fragmented Projects: A single project is scattered across multiple applications (e.g., Zotero for papers, VS Code for analysis, Google Docs for writing).
Intermixed Contexts: Each individual application becomes a clutter of unrelated projects (e.g., one Zotero library containing documents from five different research topics).
The Dual Problem of Fragmented Projects and Intermixed Contexts
Felt Problems (The Symptoms)#
These are the immediate frictions experienced by the modern knowledge worker:
Data Clutter: The tension between Fragmented Projects (a single project spread across disparate apps) and Intermixed Contexts (a single app containing noise from multiple unrelated projects).
The Filing/Naming Problem: The high cognitive overhead of deciding “where a file goes” and “what to name it” upfront.
Unsustainable Backups: Difficulty in maintaining consistent backups and the risk of Vendor Lock-in when using proprietary cloud silos.
Root Problems (The Causal Failures)#
Beneath the symptoms lie fundamental architectural flaws:
Identity Conflation: In current systems, an object’s identity is tied to its location (path-based addressing). Renaming or moving a file is effectively an act of destroying its identity.
Structural Rigidity: The “one-file-one-folder” constraint of hierarchical filesystems does not map to the graph-like nature of human understanding.
Fragile Provenance: Metadata is often volatile or implicit (loose typing), leading to loss of context during data migration.
Innumerable Document Types: The proliferation of incompatible formats (PDFs, spreadsheets, emails) prevents a unified referencing system.
Desired Features and Constraints#
The ideal knowledge management system must satisfy several intuitive and technical requirements:
The Laundry-Basket Model: The ability to “stash now, organize later,” deferring the cognitive burden of filing.
Impeccable Referencing: References must remain valid regardless of whether an object is renamed, moved, or reorganized.
Data Longevity: The system must be sustainable over a human lifespan, avoiding proprietary silos.
Semantic Search: Retrieval based on meaning rather than precise path or name recall.
The Optimization Problem#
The core challenge of Knowledge Management (KM) is often framed as a struggle between organization and retrieval. While the ultimate goal of any KM system is finding information, the act of organizing remains a necessary prerequisite. However, organization is not merely a means to an end; it is a cognitive process that can either facilitate the discovery of new knowledge or become a burdensome overhead.
To resolve this, we can model the design of a KM paradigm as a Multi-Objective Optimization (MOO) problem. We define the system’s performance based on four competing objectives:
The Filing Aspect (Cognitive Load): This addresses the “Filing Problem”—the mental energy and time required to decide where a piece of data belongs. An optimal system minimizes this effort, reducing the extraneous cognitive load on the user.
Retrieval Aspect A (Known Unknowns): This is the efficiency of pure search. It measures how quickly and reliably a user can retrieve a specific item when they know exactly what they are looking for.
Retrieval Aspect B (Unknown Knowns): This is the system’s ability to surface related items. It represents the utility of the system reminding the user, “you have encountered this concept before,” even if the specific file is not currently in focus.
Retrieval Aspect C (Serendipity/Unknown Unknowns): This is the highest form of retrieval—the ability to make robust, effortless connections between seemingly unrelated pieces of information. This is where new knowledge is synthesized.
In traditional systems, there is a strict tension between these objectives: achieving high retrieval utility (RA-A, B, and C) typically requires a high investment in manual filing. The objective of the LIST paradigm is to shift the Pareto front of this trade-off, minimizing the cost of filing (via models like the Laundry Basket) while maintaining or enhancing the utility of retrieval.
Mathematical Formulation
We seek a KM paradigm \(x\) that maximizes the following objective vector:
Where:
\(-f_{\text{filing}}\): The minimization of cognitive energy spent on organization.
\(f_{\text{RA-A}}\), \(f_{\text{RA-B}}\), \(f_{\text{RA-C}}\): The maximization of the three levels of retrieval utility.
Constraints
This optimization is subject to several non-negotiable systemic constraints to ensure the system remains viable over a lifetime:
Impeccable Referencing: \(\text{RefIntegrity}(x) = 1\). References must remain valid regardless of file movement or renaming.
Continuity of Provenance: \(\text{ProvenanceContinuity}(x) = 1\). There must be no discontinuity in the history or metadata of a file during its lifecycle.
Longevity: \(\text{Longevity}(x) \ge \text{HumanLifespan}\). The system must be decoupled from proprietary software to ensure data survives for decades.
The Hard Problem of Referencing#
While referencing is often viewed as a simple “feature” (like a hyperlink), achieving truly Impeccable Referencing is a fundamentally hard problem. The difficulty arises from the current architectural conflation of an object’s identity with its address (the file path).
Reference Granularity
To understand the hardness, we must categorize references by their granularity:
Whole-to-Whole: A reference between two top-level documents (e.g., File A $to$ File B).
Inside-to-Whole: A reference from a specific sub-section within a document to another entire document.
Whole-to-Inside: A reference from a top-level document to a specific section within another document.
Inside-to-Inside: The most granular tier, where a specific block of content refers to a specific block in another document.
The LIST Universe alongwith the 4 tiers of Referencing
Dimensions of Hardness
The “hardness” manifests in two primary technical challenges:
X-to-Inside Hardness: Referencing the interior of a document requires the document format itself to support immutable, referrable IDs for internal sections. Without this, any edit to the document’s internal structure (e.g., adding a paragraph) shifts the offsets and breaks the reference.
Read-Write Hardness: This is the paradox of referencing an object that must be written before it can be read. For example, a LaTeX file may reference a figure that is generated by a Python script. Since the image does not exist until the script runs, the system cannot assign it an immutable ID upfront, forcing a reliance on fragile mutable names.
Root Cause: Tight Coupling
The underlying cause of these problems is the tight coupling between files and their organizational layout. Because references are bound to paths rather than identities, the act of organizing data (moving or renaming files) is paradoxically the act of destroying the knowledge graph. Solving this requires decoupling the mutable name from a rigid, immutable designator.
Footnotes
Bibliography
Lilach Alon and Rafi Nachmias. Gaps between actual and ideal personal information management behavior. Computers in Human Behavior, 107:106292, June 2020. URL: https://linkinghub.elsevier.com/retrieve/pii/S0747563220300480 (visited on 2020-05-12), doi:10.1016/j.chb.2020.106292.
Krisztian Balog and Tom Kenter. Personal knowledge graphs: A research agenda. In Proceedings of the ACM SIGIR International Conference on the Theory of Information Retrieval (ICTIR). 2019.
Daniele Di Sarli and Filippo Geraci. GFS: a Graph-based File System Enhanced with Semantic Features. In Proceedings of the 2017 International Conference on Information System and Data Mining, 51–55. Charleston SC USA, April 2017. ACM. URL: https://dl.acm.org/doi/10.1145/3077584.3077591 (visited on 2024-09-03), doi:10.1145/3077584.3077591.
Jesse David Dinneen and Charles-Antoine Julien. The ubiquitous digital file: A review of file management research. Journal of the Association for Information Science and Technology, 71(1):E1–E32, 2020. URL: https://asistdl.onlinelibrary.wiley.com/doi/abs/10.1002/asi.24222 (visited on 2020-05-12), doi:10.1002/asi.24222.
Matthias Geel. Memsy: A Personal Resource Management Infrastructure. PhD thesis, ETH Zurich, 2015. URL: http://hdl.handle.net/20.500.11850/113707 (visited on 2020-05-12), arXiv:20.500.11850/113707, doi:10.3929/ETHZ-A-010603237.
Julien Gori, Han L. Han, and Michel Beaudouin-Lafon. FileWeaver: Flexible File Management with Automatic Dependency Tracking. In Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology, UIST '20, 22–34. New York, NY, USA, October 2020. Association for Computing Machinery. URL: https://dl.acm.org/doi/10.1145/3379337.3415830 (visited on 2026-04-27), doi:10.1145/3379337.3415830.
Stefan Gruner. Heinz zemanek's almost forgotten contributions to the early philosophy of informatics. In Australasian Conference on Information Systems. 2016.
William Kent. Data and Reality 2nd Edition. 1st Books Library, Bloomington, Indiana, rev edition, 2000. ISBN 978-1-58500-970-1.
Saul A. Kripke. Naming and Necessity. Harvard Univ. Press, Cambridge, Mass, 1980. ISBN 978-0-674-59846-1 978-0-674-59845-4.
Luiz Fernando Lemos, Diego Castro, Carlos Eduardo Pantoja, and Bruno Freitas. Tag-Driven File Management System. Revista Eletrônica de Iniciação Científica em Computação, 24:155–161, March 2026. URL: https://journals-sol.sbc.org.br/index.php/reic/article/view/6680 (visited on 2026-06-16), doi:10.5753/reic.2026.6680.
Darrell E. Long and Ethan L Miller. Dynamic Non-Hierarchical File Systems for Exascale Storage. Technical Report DE-FC02-10ER26017/DE-SC0005417, 1170868, University of California, Santa Cruz, February 2015. URL: http://www.osti.gov/servlets/purl/1170868/ (visited on 2023-08-02), doi:10.2172/1170868.
Tony Mason. Laundry Baskets: The New File System Namespace Model. September 2021. URL: https://fsgeek.ca/2021/09/28/laundry-baskets-the-new-file-system-namespace-model/ (visited on 2023-08-02).
William Anthony Mason. Indaleko : The Unified Personal Index. PhD thesis, University of British Columbia, 2025. URL: https://doi.library.ubc.ca/10.14288/1.0449905 (visited on 2025-12-21).
Eric Miller. The Problem with File Systems. April 2017. URL: https://sosheskaz.github.io/technology/2017/04/14/Filesystems-Suck.html (visited on 2025-12-13).
Markus Schröder, Christian Jilek, and Andreas Dengel. A Human-in-the-Loop Approach for Personal Knowledge Graph Construction from File Names. In CEUR Workshop Proceedings. 2022.
Margo Seltzer and Nicholas Murphy. Hierarchical File Systems are Dead. In Usenix. 2009.
Beat Signer and Moira C. Norrie. As We May Link: A General Metamodel for Hypermedia Systems. In Christine Parent, Klaus-Dieter Schewe, Veda C. Storey, and Bernhard Thalheim, editors, Conceptual Modeling - ER 2007, volume 4801, pages 359–374. Springer Berlin Heidelberg, Berlin, Heidelberg, 2007. URL: http://link.springer.com/10.1007/978-3-540-75563-0_25 (visited on 2026-04-27), doi:10.1007/978-3-540-75563-0_25.
Beat Signer, Reinout Roels, Robert van Barlingen, and Brent Willems. Back to the Future: Bringing Original Hypermedia and Cross-Media Concepts to Modern Desktop Environments. In Proceedings of the 32nd ACM Conference on Hypertext and Social Media, HT '21, 277–282. New York, NY, USA, August 2021. Association for Computing Machinery. URL: https://dl.acm.org/doi/10.1145/3465336.3475122 (visited on 2026-04-27), doi:10.1145/3465336.3475122.
Eno Thereska, Oriana Riva, Richard Banks, Sian Lindley, and Richard Harper. Beyond file systems: understanding the nature of places where people store their data. Technical Report, MicrosoftResearch, 2013.
Valter Uotila, Jiaheng Lu, Dieter Gawlick, Zhen Hua Liu, Souripriya Das, and Gregory Pogossiants. MultiCategory: Multi-model Query Processing Meets Category Theory and Functional Programming. Proceedings of the VLDB Endowment, 14(12):2663–2666, July 2021. URL: http://arxiv.org/abs/2109.00929 (visited on 2025-11-13), arXiv:2109.00929, doi:10.14778/3476311.3476314.
Karl Voit. TagTrees: Improving Personal Information Management Using Associative Navigation. PhD thesis, Graz University of Technology, 2012.
Karl Voit, Keith Andrews, and Wolfgang Slany. Why Personal Information Management (PIM) Technologies Are Not Widespread And What to do About It. In PIM Workshop, 5. 2009.
Heinz Zemanek and International Federation for Information Processing, editors. The Skyline of Information Processing: Proceedings of the 10th Anniversary Celebration of the IFIP, Amsterdam, Oct. 25, 1970. North-Holland, Amsterdam, 1972. ISBN 978-0-7204-2071-5.
Remzi H. Arpaci-Dusseau and Andrea C. Arpaci-Dusseau. Operating Systems: Three Easy Pieces. CreateSpace Independent Publishing Platform, Erscheinungsort nicht ermittelbar, 2023. ISBN 978-1-9850-8659-3.
Joep Meindertsma. Atomic Data. URL: https://docs.atomicdata.dev/ (visited on 2025-12-25).
M-Files. 2019 Intelligent Information Management Benchmark. White Paper. 2019. URL: https://www.project-consult.de/wp-content/uploads/2019/04/M-Files_IIM_Benchmark_Report_2019.pdf (visited on 2026-04-27).
Nayuki. Designing better file organization around tags, not hierarchies. March 2017. URL: https://www.nayuki.io/page/designing-better-file-organization-around-tags-not-hierarchies#contents (visited on 2020-05-12).