Order Matters: Sequence to sequence for sets

Editorial record

Plain-language summary

Many problems involve sets rather than sequences, yet models were trained by picking an arbitrary order. The authors show experimentally that this arbitrary choice can swing results, for example when sorting numbers or parsing. They introduce a set encoder that reads all elements, processes them with attention over a memory for several steps, and writes outputs with a pointer network. The paper is an early, concrete argument that inductive biases about ordering and permutation invariance matter, a theme that reappears in graph networks and in positional encodings for transformers.