Date of Award
5-31-2026
Document Type
Campus Access Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Computer Science
First Advisor
Dan Simovici
Abstract
@font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-536870145 1107305727 0 0 415 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-unhide:no; mso-style-qformat:yes; mso-style-parent:""; margin:0in; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman",serif; mso-fareast-font-family:"Times New Roman";}.MsoChpDefault {mso-style-type:export-only; mso-default-props:yes; font-family:"Aptos",sans-serif; mso-ascii-font-family:Aptos; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:Aptos; mso-fareast-theme-font:minor-latin; mso-hansi-font-family:Aptos; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;}.MsoPapDefault {mso-style-type:export-only; margin-bottom:8.0pt; line-height:115%;}div.WordSection1 {page:WordSection1;}
Large language models (LLMs) have become pervasive in computational research and practice, yet fundamental questions about their intrinsic properties — the structural, semantic, and dynamical characteristics that govern what they produce and what they can do — remain incompletely addressed. Existing evaluation paradigms focus primarily on task-specific performance, leaving open the question of what intrinsic, theoretically grounded properties large language models possess and how those properties manifest across generation, reasoning, and iterative transformation. This dissertation addresses that question through a unified research program spanning four interconnected dimensions: the information-theoretic structure of LLM outputs, their semantic organization, their reasoning boundaries, and their dynamical behavior under iterative self-transformation.
@font-face {font-family:"Cambria Math"; panose-1:2 4 5 3 5 4 6 3 2 4; mso-font-charset:0; mso-generic-font-family:roman; mso-font-pitch:variable; mso-font-signature:-536870145 1107305727 0 0 415 0;}p.MsoNormal, li.MsoNormal, div.MsoNormal {mso-style-unhide:no; mso-style-qformat:yes; mso-style-parent:""; margin:0in; mso-pagination:widow-orphan; font-size:12.0pt; font-family:"Times New Roman",serif; mso-fareast-font-family:"Times New Roman";}.MsoChpDefault {mso-style-type:export-only; mso-default-props:yes; font-family:"Aptos",sans-serif; mso-ascii-font-family:Aptos; mso-ascii-theme-font:minor-latin; mso-fareast-font-family:Aptos; mso-fareast-theme-font:minor-latin; mso-hansi-font-family:Aptos; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi;}.MsoPapDefault {mso-style-type:export-only; margin-bottom:8.0pt; line-height:115%;}div.WordSection1 {page:WordSection1;}
The first contribution establishes compression rate as a theoretically grounded and empirically validated proxy for mineability — the capacity of a text to support structured information extraction. Drawing on Shannon's information theory and the connection between compressibility, entropy, and predictability, we demonstrate a consistent inverse correlation between compression rate and perplexity across varying generation parameters, prompt types, and n-gram orders. This finding establishes a lightweight, model-agnostic framework for text quality assessment that requires no task-specific models or human annotation.
The second and third contributions introduce Semantic-Aware Association Rule Mining (SA-ARM), a framework that integrates LLMs into classical association rule mining pipelines to overcome the semantic brittleness of exact-match symbolic methods. The framework performs LLM-based item normalization, ontology-free semantic grouping, and a formally derived support normalization that corrects for abstraction-induced statistical bias. We demonstrate that concept-level rules produced by SA-ARM dramatically outperform item-level baselines in coverage, interpretability, and robustness, while the support normalization strategy ensures statistical comparability across abstraction levels — a contribution that, to our knowledge, has not been previously addressed in the literature.
The fourth contribution develops a systematic evaluation framework for LLM reasoning on graph-structured problems, grounded in computational complexity theory. By organizing evaluation tasks into a five-tier hierarchy — from formula-based construction through algorithmic search, constrained generation, weighted optimization, and NP-hard multi-constraint problems — and isolating optimization as a distinct reasoning bottleneck through controlled task pairs, we characterize the boundaries of LLM reasoning capability in theoretically meaningful terms. A consistent 28–67% performance drop when optimality constraints are added identifies optimization as a fundamental barrier rooted in the tension between autoregressive generation and global optimality requirements.
The fifth contribution investigates LLM behavior under iterative expansion-summarization transformations through a large-scale empirical study spanning five frontier models, six content domains, and 245 experimental configurations totaling 12,855 iterations. We demonstrate universal convergence dynamics alongside model-specific equilibrium states, characterized by stable compression rates and high embedding consistency (94.2–98%) despite substantial surface variation (26.7–71%). Cross-model embedding similarity at equilibrium (76.3%, mean of all pairwise model combinations) is substantially lower than within-model similarity (96.0%), demonstrating that models encode identical content through distinct, training-dependent representational strategies. Contrasting scaling laws across model families further reveal that behavioral properties do not scale uniformly, challenging assumptions inherited from benchmark-focused scaling research.
Taken together, these contributions establish that large language models possess measurable, theoretically interpretable structural and behavioral properties — and that their outputs are not arbitrary artifacts of generation but objects amenable to principled analysis. The dissertation advances foundations for principled use of LLMs in knowledge discovery pipelines and for systematic behavioral analysis, with implications for model selection, quality assessment, and the design of LLM-integrated computational systems.
Recommended Citation
Saadat, Cyrus, "Structure, Semantic, and Dynamics in Large Language Models: Foundations for Knowledge Discovery and Behavioral Analysis" (2026). Graduate Doctoral Dissertations. 1158.
https://scholarworks.umb.edu/doctoral_dissertations/1158
Comments
Free and open access to this work is made available to the UMass Boston community by ScholarWorks at UMass Boston. Those not on campus and those without a UMass Boston campus username and password may gain access to this work through Interlibrary Loan. If you have a UMass Boston campus username and password and would like to download this work from off-campus, click on the “Off-Campus Users” button.