High Quality Embeddings for Horn Logic Reasoning
Quick Answer
This paper presents novel methods for generating high-quality embeddings for horn logic reasoning, utilizing triplet loss to enhance the efficiency of neural networks in logical searches.
Quick Take
This paper presents novel methods for generating high-quality embeddings for horn logic reasoning, utilizing triplet loss to enhance the efficiency of neural networks in logical searches. Key innovations include improved anchor generation and balanced example selection, leading to better performance across various knowledge bases.
Key Points
- Introduces triplet loss for training embeddings in logical reasoning tasks.
- Focuses on generating anchors with repeated terms for better performance.
- Balances easy, medium, and hard examples during training for robustness.
- Conducts experiments comparing embeddings across various knowledge bases.
- Aims to identify characteristics of effective embeddings for reasoning tasks.
Paper Resources
Article Excerpt
From source RSS / original summaryarXiv:2605. 20467v1 Announce Type: new Abstract: Neural networks can be trained to rank the choices made by logical reasoners, resulting in more efficient searches for answers. A key step in this process is creating useful embeddings, i. e. , numeric representations of logical statements. This paper introduces and evaluates several approaches to creating embeddings that result in better downstream results.
We train embeddings using triplet loss, which requires examples consisting of an anchor, a positive example, and a negative example. We introduce three ideas: generating anchors that are more likely to have repeated terms, generating positive and negative examples in a way that ensures a good balance between easy, medium, and hard examples, and periodically emphasizing the hardest examples during training.
We conduct several experiments to evaluate this approach, including a comparison of different embeddings across different knowledge bases, in an attempt to identify what characteristics make an embedding well-suited to a particular reasoning task.
Want this in your inbox every morning?
Daily brief at your local 8am — bilingual EN/中文, free.
More from arXiv cs.AI
See more →Procedural Memory Distillation: Online Reflection for Self-Improving Language Models
Procedural Memory Distillation (PMD) enhances reinforcement learning by converting cross-episode signals into reusable memory, improving Qwen3-8B and OLMo3-Instruct-7B models by 3.8-5.5% on SCIKNOWEVAL and 7.9-13.6% on . The co-evolution of policy and memory allows for more effective self-supervision, demonstrating significant performance gains when both components are active.