Software Engineering Papers

ALT Code embedding models have become a key building block for many software engineering applications, yet their internal representations remain poorly understood. Existing evaluations tend to focus on downstream task performance, offering limited insight into whether a single embedding can support both functional abstraction and implementation-sensitive analysis. To address this gap, we present a large-scale empirical study of 14 state-of-the-art code embedding models. We evaluate their representational behavior through three complementary analyses: supervised probing for functional abstraction and implementation fidelity, zero-shot retrieval experiments to assess geometric similarity, and controlled perturbation experiments that measure embedding responses to semantics-preserving code transformations. The results reveal a consistent abstraction-fidelity tension: models that better preserve stylistic and structural signals tend to do so at the expense of semantic generalization, while mod

ALT A negative reachability verdict removes a finding from human review. We check such verdicts against execution. Our pipeline, SourceLens, maps Semgrep findings to a Python program's call graph, whose edges a query takes from the calls that Joern links in its code property graph. We ran two command-line programs from the PyCG macro-benchmark, in which the benchmark's manual call graph and Joern disagree, under coverage and recorded the calls between their functions. On these edges, 123 functions of asciinema are unreachable from the program's entry points. The project's own integration script executes 86 of them, and all our runs 113. In Sublist3r, 19 of 24 are executed. The latest Joern release gives the same verdicts. The missing calls go through values, such as classes and functions passed around, or have no call expression, as in with blocks and property reads. In asciinema, four missing edges make 50 of the 113 executed functions reachable. The manual call graphs also lack observed

ALT Code language models complete, review, and repair programs without exposing the program structure encoded in their hidden states. In this paper, we test whether the hidden states of a code model can be described and edited using the information a compiler front end computes. For each token of a Python program, standard analyzers provide its position in the abstract syntax tree, its scope, its definition-use links, and its syntactic category. We fit a decomposition that expresses every hidden state of a frozen model as a sum of one term per token, each combining the token with this structural label, and we replace the model's own states with the decomposition at every layer while it predicts the output of a small program. If the model still answers correctly, the decomposition preserves the information the model uses. We apply this test to 14 open models, seven of them trained for code, and compare against a baseline decomposition of the same size built from token positions. The static-

ALT AI coding agents now do long, multi-step and multi-session software work with little supervision. When a developer comes back to that work, the useful question is not what the model said but what was actually done, by whom, and whether it is finished. We study this in three parts. First, we verify Agenttrail, an open-source local observability tool for coding agents, against its source at commit e4ba2da: nine documented constants and mechanisms hold at file-and-line level, two event pipelines are never unified, and its Map service has an unauthenticated local mutation surface. Second, we run a prototype of our system on 45 real coding-agent sessions (102,233 records, 20,577 tool executions, 2,336 human-to-agent episodes). With a blind-audited classifier (precision 83.3% and 96.7%) we estimate that 23.4% (95% CI 17.1 to 35.7) of code-editing episodes end with no independent check after the last edit, and that 57.8% of to-dos the agent marked complete after editing had no proof in betwee

ALT Large language models (LLMs) have shown strong performance on software engineering (SE) tasks, driving the need for high-quality datasets for reliable training and evaluation. SWE-bench-style datasets (SWE datasets) are widely used to evaluate LLM-based coding agents on real-world software issues, where task clarity and test coverage directly affect whether instances are solvable and verifiable. However, quality issues persist in existing SWE datasets, and manual annotation is costly and labor-intensive, motivating LLM-based annotation. Existing approaches typically rely on static, overly general rubrics, limiting their ability to handle diverse and nuanced scenarios and often yielding high false positive rates. In contrast, human annotators draw on accumulated experience to make context-aware judgments. Inspired by this, we propose ExLabel, an LLM-based, self-evolving, experience-guided assistant for task clarity and test coverage assessment in SWE datasets. ExLabel maintains an evolv

ALT Quantum software engineering (QSE) requires understanding application requirements, quantum programs and algorithms, quantum computation paradigms, executions, and deployment. However, relevant quantum computing (QC) concepts are often defined across physical, mathematical, and paradigm-specific levels of abstraction, making it difficult for the QSE community and hence proposed methods to refer to them consistently. To this end, we present QSoftCore, a conceptual model that reorganizes established QC concepts from a software engineering perspective and connects quantum applications, computational problems, algorithms, configurations, programs, and paradigm-specific computation specifications with state evolution, execution results, and backends. Our assessment against QSE literature across key software engineering activities indicates that QSoftCore provides shared QC referents for quantum computations, executions, deployment configurations, backends, and measurement evidence across th

ALT Continuous-integration evidence is conditional: a workflow determines which jobs should run, while an archive records which jobs remain observable. Reconstructing requirements from those observations lets an omission erase its own obligation. We present a qualification method that compiles a pinned workflow into conditional job requirements independently of target presence, retains a fixed capture inventory, and checks source-complete derivation certificates with a separate implementation. For a fixed source, context, scope and binding universe, with sound obligations and applicability judgments that preserve established truth under information extension, withdrawing observation facts cannot create authorization. The implementation handles static matrices, dependency guards and bounded pinned reusable workflows, preserving unknown applicability and separating tolerated outcomes from evidence coverage. Both implementations match the 45 hand-specified authorization controls, reject 180 s

ALT Empirical studies of polyglot repositories apply the same analyses to every language of a corpus, such as separating code from comments, computing metrics, normalizing identifiers, and screening for duplicated code. Assembling them from specialized tools introduces differences in language coverage, measurement conventions, and workflows. We present PYREPRISM, an open-source Python library and command-line tool that separates language-specific lexical processing from reusable analyses: each language declares its lexical surface, which yields a lossless stream of typed tokens, and every analysis is implemented once on that stream. Across 187 built-in languages, language-specific declarations have a median length of 16 lines, and no language overrides a shared analysis. Against a reference parser, comment and string classification reach F1 scores of 0.994 and 0.958 on 1,412 files in 12 languages that were never used to develop the tokenizer. In a study of a polyglot repository, one tool r

ALT Context. Video ads reach streaming and connected-TV apps through VAST documents, and companies count ads from the HTTP requests the player's client library sends. A tracking error can leave playback intact and surface later as disagreeing counts. Objective. To measure how far open-source VAST clients meet the player duties of VAST 4.3, whether verdicts depend on the runtime, what generated inputs add, and when deviations were introduced. Method. A catalog of 108 requirements was extracted from the specification. Six open-source clients in eight configurations ran against 69 crafted test vectors and the IAB's sample tags, under Node.js and in five browsers on three engines, with every request recorded. Grids, fuzzing and a sweep over 36 releases extend the catalog; Google's closed IMA SDK is a comparison. Results. The open-source clients have 78 findings and 3 to 21 must-level deviations per configuration. None of the tested open-source clients that follow redirect (Wrapper) chains fill

ALT Large language models pass most functional-correctness benchmarks, but passing tests says little about whether their code is readable, efficient or secure. Many techniques claim to improve these qualities, yet their evidence is scattered, usually covers one quality dimension and is rarely compared at an equal compute budget. We contribute a systematic review and a controlled re-evaluation. A systematic literature review of 628 primary studies (July 2021 to October 2026), with disclosed, author-reviewed LLM assistance, organizes the techniques by stage, signal source and quality target. Most studies measure only the dimension they target, gains often come with unchanged or unreported correctness, single automated quality signals are gamed, and compute, variance and contamination are rarely reported. A controlled re-evaluation runs ten techniques on four models at matched budgets on correctness, maintainability, efficiency and security benchmarks, extends it to thirteen current models an