Manolios and Fanelli (1994)

use Elman's first-order architecture ((Elman, 1990)), as Cleeremans et al. (1989) , but train it to classify strings in small learning sets as grammatical or ungrammatical --according to four of the grammars in Tomita (1982) -- using backpropagation through time (Rumelhart et al., 1986) with batch updating (see section 3.4.1). Once the DTRNN has been trained, automata are extracted using a special clustering algorithm that has been described in section 5.4. One of the most interesting features of the paper is its graphical study of the learning process, and , in particular, that of the size of clusters as training progresses. The size of the clusters is proposed as an indicator of generalization ability.