Less can be more for predicting properties with large language models

Predicting properties from coordinate–category data—sets of vectors paired with categorical information—is fundamental to computational science, manifesting in materials science as predicting formation energies or elastic moduli from crystal structures. While large language models (LLMs) have increasingly been applied to such tasks, optimal strategies for achieving reliable predictions remain elusive. Here we report fundamental limitations in LLM’s ability to learn from coordinate information in coordinate–category data.

Through systematic experiments using synthetic datasets with tunable…

Probing LLMs with coordinate–category information mixtures To understand how LLMs perform in predicting data containing both coordinate and category information, we created synthetic datasets. Each data point consists of 3D coordinates (coordinate in... [28868 chars]