Inconsistent behaviour when importing grammars with lexical rules and modes

[akosthekiss]

There seems to be an inconsistent behaviour when importing grammars with lexical rules and modes. ANLTR accepts some variations and rejects others. The documentation does not mention such restrictions (or at least not clearly).

Try 1: All lexical rules in the global mode (OK)

lexer grammar ImpALexer;
import ImpBLexer;
X: 'x';
Y: 'y';

lexer grammar ImpBLexer;
X: 'a';
Y: 'b';

% java -jar antlr-4.13.2-complete.jar ImpALexer.g4

Try 2: Overridden rule in a named mode, overriding rule in the global mode (OK)

Put a lexical rule in a mode in the imported grammar:

lexer grammar ImpBLexer;
X: 'a';
mode M;
Y: 'b';

% java -jar antlr-4.13.2-complete.jar ImpALexer.g4

Try 3: Both overridden and overriding rules in a named mode (FAIL)

Put the same lexical rule in the same mode in the main grammar:

lexer grammar ImpALexer;
import ImpBLexer;
X: 'x';
mode M;
Y: 'y';

% java -jar antlr-4.13.2-complete.jar ImpALexer.g4
error(51): ImpALexer.g4:4:0: rule Y redefinition; previous at line 5

This also happens if the names of the modes differ (e.g., change name of mode to N in ImpBLexer).

Try 4: Overridden rule in global mode, overriding rule in named mode (FAIL)

Change the imported grammar ImpBLexer back to its original version without any modes, but keep the main grammar with mode M.

% java -jar antlr-4.13.2-complete.jar ImpALexer.g4
error(51): ImpALexer.g4:5:0: rule Y redefinition; previous at line 3

Analysis

So, it seems that only a lexical rule in the global mode can override an imported rule (but in that case it does not matter whether it was in the global mode or in a named mode).

Note: The errors seem to refer to incorrect files and/or incorrect line numbers.

I am not sure whether the current behaviour is because of some constraints (theoretical, design, or implementation), or just a corner case not handled right. It would be nice to either fix this or document the restrictions.