Citation:
Eric Allen Engle,
Smoke and Mirrors or Science? Teaching Law with Computers -- a Reply to Cass Sunstein on Artificial Intelligence and Legal Science,
9 Rich. J.L. & Tech. 9 (2003). 

Change Font Face | Change Font Size | Change Background Color color | Change Text Color color | Change Link Color | Change Visited Link Color
Copyright © Eric Allen Engle

*1 The application of computer science in the law has largely, and productively, centered on educational programs 1 and programs generating and managing databases and data management. Some limited work, however, has been done in the use of artificial intelligence ("AI") to present models 2 of legal decision-making. 3 The majority of the work involving AI in the law, as the majority of work in AI generally, has focused on developing expert systems. 4 An expert system attempts to solve one problem, or one class of problems well and should be distinguished from general systems, which seek to solve any problem. While databases and didactic applications involve the most productive work being done, AI presents more complex and more interesting problems having the potential to further increase productivity. Therefore, while AI may still be in its infancy (as was all of computer science until twenty years ago), its intellectual vistas are potentially limitless.

*2 It was thus with mixed reactions that I read a recent article by Cass Sunstein, arguing that AI in law is of very limited utility and, at present, can only solve very limited classes of legal problems. Sunstein likens computer AI in law at present to a glorified LEXIS. 5 This is not the case because AI does more than automatically sort data. Sunstein argues that computers cannot reason analogically because a computer cannot make an evaluative 6 (moral) choice 7 and thus cannot infer new rules of production from old rules of production. Sunstein argues that because computers cannot reason analogically, they cannot develop AI, 8 at least not yet. 9 As we shall see, this position is simply untenable. 10

*3 There is some cause to focus, as Sunstein appears to, 11 on expert systems rather than general systems: the problems facing the elaboration of an expert system are much more manageable than those facing the development of a general system. However, AI programs are not simply split between expert systems and general systems but also between "static" systems with fixed (pre-programmed) rules of production and "dynamic" systems which develop their own rules of production based upon experience. 12

*4 Static AI programs, such as the one I present here, use invariable pre-programmed rules of production. They do not learn because their rules of production are "fixed." For example,
though chess programs are the most often-cited example of successful AI, they are, in fact, an example of "static" AI (albeit highly-developed). This is because most chess programs do not adapt their algorithms to take advantage of the weaknesses of their opponent, but rather use a pre-programmed algorithm.

On the other hand, "dynamic" computer programs generate their own rules of production and are consequently 'trainable." 13 Programs which "learn" by dynamically developing rules of production do exist 14 and truly "learn." Such programs must derive principles (for example, the shape of a letter) from cases (a given set of experiences that are either provided by the user or are pre-programmed)--which is the very type of reasoning that Sunstein argues a computer cannot do. 15

*5 The most well-known work in AI that actively "learns" is in the field of neural networks. 16 Neural networks are best known for being used to recognize characters. This field of AI is well-developed and, therefore, no longer in its infancy. Programs such as Apple's Inkwell do indeed "learn" to recognize characters and words depending on input and correction by the user. 17

*6 Rather than focusing on the split between "active" programs, which "learn," and "static" programs, which do not, Sunstein focuses on a split between analogical reasoning (inductive logic) and deductive reasoning. Deductive reasoning is necessarily true but develops no new propositions. Inductive reasoning is not necessarily true but develops new propositions. Sunstein argues that computer programs, or at least computer programs in law, cannot learn to properly apply analogical reasoning 18 because he argues that they cannot derive principles from cases-which, as this article will show, is false. 19 Algorithms for deriving new principles from existing cases do exist 20 and usually invoke iterative and/or pseudo-random functions or abduction to do so. As Aikenhead notes, "[n]umerous systems have been constructed that simulate aspects of legal analogizing" 21 including SHYSTER, a case-based legal expert system. 22 Aikenhead's arguments are much more refined and computationally accurate 23 than Sunstein's.

*7 Programmatic representations of inductive inference, like most computational problems, are, in fact, trivial 24 (in the computational sense of the word). For example, presume that we have three known cases, each having three attributes with the following values:
 

CASE        VALUES

----------------------  
I           1,2,3
II          A,B,C
III         A,2,3
 
Given a Case IV, with values 1, 2, C, analogical reasoning would compare the values and determine that Case IV is most similar to Case I and apply the same rule in Case IV.

*8 Inductive amplification does not merely apply existing rules by analogy to a new case. Rather, it derives a new rule from existing cases. It is this type of reasoning which Sunstein argues a computer is incapable of doing. Using the same example, the derivation of a new rule would be that any new case will be decided according to the rule used in the case with the greatest number of similar elements. Such a rule could be initially programmed into the computer as a "static" rule of production, or could be "learned" through trial and error as a dynamic rule of production.

*9 Let us consider the example of a node in a neural network with three inputs that is suddenly given a new fourth input. The new input must be tested and controlled to determine whether the new input is to favor or disfavor a certain outcome and to determine what weight should be applied to the new input. That control can either be pre-programmed or, more efficiently, be supplied by the user. Statically, the program would be given at least two rules of production. Say that new factors shall have, for example, 1/n weight (where n equals the total number of factors to be considered) and that factor n (the new factor) should favor the party with no burden of proof unless instructed otherwise. To these rules of production could be added a third human "oversight" function: by running, for example, three trials where the human oversees the outcome, verifying or denying the computer response, and programmatically applies the rule of production that the human taught the computer, the computer would be able to correctly apply the new rule of production and apply it simulating analogical reasoning. Such a control function could be pre-programmed, however, by iterating 25 through all combinations, first favoring, then disfavoring, the outcome and using different weights and comparing the result to existing cases.

*10 If Sunstein's position is, so far as programming computers goes, untenable, what about its validity in law? There are valid grounds for questioning Sunstein's assumptions about law. Legal scholars should recognize that, although the common law relies heavily on analogical reasoning, the civil law relies equally heavily on deductive reasoning. 26 Sunstein would have us ignore this. 27 Since Sunstein does not appear to question the ability of a program to represent deductive reasoning, it seems his claim that it is impossible, at present, for a program to represent the law is ill-considered.
*11 Sunstein hedges his position. First, he argues that his position that an analogical inference engine is impossible cannot, in fact, be verified, 28 which takes his opinion out of the realm of science. Second, Sunstein argues that "things might change." 29 These hedges appear to be contradictory. Sunstein has said that his claim cannot be verified or falsified, yet implies that new technology may one day permit machines to model analogical reasoning. 30 Even if we could grant both these (contradictory) hedges, Sunstein's argument goes too far and can be demonstrated to be erroneous using a trivial example of analogical reasoning.

*12 Let us presume that two Cases with three elements each will be similarly decided if two of the elements are the same (that is, if the two cases are analogically similar). Programmatically, analogical reasoning could be represented thus:

//Assign values to our known case
Case1.Argument1 = 1;
Case1.Argument2 = 1;
Case1.Argument3 = 1;
Case1.Outcome = "Not Guilty";
//Initialize our unknown case
Case2.Outcome = "Unknown";
//Trivial example of analogical reasoning
if (case2.Argument1 = case1.Argument1)
{
if (Case2.Argument2== case1.Argument2)
{
Case2.Outcome == Case1.Outcome;
}
if (Case2.Argument3== case1.Argument3)
{
Case2.Outcome ==Case1.Outcome;
}
}
if (Case2.Argument2 == Case1.Argument2)
{
if (Case2.Argument3== Case1.Argument3)
{
Case2.Outcome == Case1.Outcome;
}
}
Alternatively, this could be represented using weighted balances rather than binary values, for example, presuming the plaintiff has the burden of proof:
If (plaintiff.arguments[weight] >defendant.arguments[weight])
{
decisionForPlaintiff();
}
*13 These reasons give us cause to question Sunstein, both as programmer and legal theorist. Sunstein would have done better to argue that computation of law is impossible because building and training a neural network to operate as a general system is too complex. Such an argument might hold water. Complexity in programming a general system of AI can be found in the fact that creation of a general system would first require a natural language parser. However, such parsers have existed for at least twenty years, and are constantly improving. Presuming that an adequate parser could be found or built, the next difficulty would be developing self-learning neural networks. This is difficult, but certainly not impossible. Such a neural network should include a human "control" to test and correct the engine's inferences, as that would generate a better engine more quickly than one based purely on pre-programmed rules of production. The next difficulty would be to generate a sufficiently large field of cases to teach the engine. Again, this is not an insurmountable task. Though this task would, like adapting a natural language parser, be time-consuming. A final difficulty might be the computational speed or memory required. This is the least of the problems, as enormous expert systems are well within the computational power of contemporary desktop computers. Although general systems in AI are the exception, they are not computationally impossible and would not require some new break through in microprocessor architecture or memory 31 as Sunstein argues.

*14 This contentious introduction is intended to set the stage for my very unambitious static inference engine. I have written a program to assist in the teaching (and perhaps practice) of tort law. This program (unlike my next) is well within the hedges that Sunstein places on his bets. It does not use a neural network, and does not "learn" a new system of rules (but it could have by relying on either a neural-node model or using a tree model with either head or tail recursion). Instead, the program provided is "static." I have given a set of rules of production (which represent the basics of tort law) and then the program asks the user a series of questions to determine whether a tort has taken place and giving reasons for its decision. The program essentially uses a series of "binary" tests as rules of production.

*15 Another approach to static AI modeling of law (which I used in a program to determine the tax residence of a person according to the French-U.S. tax treaty 32) uses a weighted "balancing" approach. One advantage to the binary method is that when the law can be represented as "either/or," it is highly methodologically defensable as a model of law. In contrast, the law only rarely quantifies the weight given to its nodes when balancing competing interests. If there were any computational problems in generating AI models of law, it is, in my opinion, here: the quantification of factors that may be quantifiable but generally are not quantified by courts. Even this difficulty is not insurmountable. The law sometimes uses quantifiable economic data to calculate the weight of factors in legal balancing tests. So, using a " balancing" test is computationally and legally defensible.

*16 The program is basically self-explanatory and includes a brief essay on tort law, which, along with the source code, is reproduced below. The program was written in xTalk, a derivative of Pascal, using the MetaCard engine. I chose xTalk rather than C or a C- derived language because it is "pseudo" English and because the auto-formatting of the metaTalk editor makes the source code readable even for non-programmers. Given the current computational speed and power of CPUs, I see no real advantage to developing AI using a lower-level language such as C or assembler. However, the fact that there are plenty of C AI libraries explains why programmers may wish to port them to Pascal or xTalk, so that non-programmers can correctly assess the utility of AI in representing the law for purposes of teaching, research, and even legal practice. A great deal of useful work can be done using computers to model law either in education or data management or in legal practice including, eventually, general AI systems. Whether the position that Professor Sunstein takes is defensible will be revealed in time.

Notes: a1. Eric Engle, J.D., D.E.A., L.L.M., teaches at the University of Bremen (Lehrbeauftagter) and is also a research fellow at Zentrum Fuer Eurpaeische Rechtspolitik.

1. See, e.g., Dan Hunter, Teaching Artificial Intelligence to Law Students, 3 Law Tech. J. 3 (Oct. 1994), available at http://www.law.warwick.ac.uk/ltj/3-3h.html (discussing the methodological problems involved, especially the problems of developing sylabi for teaching law and AI).

2. See Alan L. Tyree, Fred Keller Studies Intellectual Property, at http://www.law.usyd.edu.au/~alant/alta92-1.html (last modified Dec. 20, 1997) (discussing self-paced instructional programs).

3. See, e.g., Alan L. Tyree, FINDER: An Expert System, at http://www.law.usyd.edu.au/~alant/aulsa85.html (last modified Dec. 20, 1997).

4. See, e.g., Carol E. Brown and Daniel E. O'Leary, Introduction to Artificial Intelligence and Expert Systems, at http://accounting.rutgers.edu/raw/aies/www.bus.orst.edu/faculty/brownc/es_tutor/es_ tutor.htm (last modified 1994).

5. "My conclusion is that artificial intelligence is, in the domain of legal reasoning, a kind of upscale LEXIS or WESTLAW-- bearing, perhaps, the same relationship to these services as LEXIS and WESTLAW have to Shepherd's." Cass R. Sunstein, Of Artificial Intelligence and Legal Reasoning 7 (Chicago Public Law and Legal Theory Working Paper No. 18, 2001), available at http://www.law.uchicago.edu/academics/publiclaw/resources/18.crs.computers.pdf.

6. "I have emphasized that those who cannot make evaluative arguments cannot engage in analogical reasoning as it occurs in law. Computer programs do not yet reason analogically." Id. at 9.

7. "I believe that the strong version [of claims for machine-based AI] is wrong, because it misses a central point about analogical reasoning: its inevitably evaluative, value-driven character." Id. at 5.

8. "Can computers, or artificial intelligence, reason by analogy? This essay urges that they cannot, because they are unable to engage in the crucial task of identifying the normative principle that links or separates cases." Id. at 2. Sunstein appears to be grappling with the so-called law of Hume (the idea that moral choice cannot be deduced, but is instead arbitrary), yet does not address Hume. E.g., "[S]ince HYPO can only retrieve cases, and identify similarities and differences, HYPO cannot really reason analogically. The reason is that HYPO has no special expertise is [sic] making good evaluative judgments. Indeed, there is no reason to think that HYPO can make evaluative judgments at all." Id. at 6.

9. "[W]e cannot exclude the possibility that eventually, computer programs will be able both to generate competing principles for analogical reasoning and to give grounds for thinking that one or another principle is best." Id. at 8.

10. "A trained [neural] network is capable of producing correct responses for the input data it has already 'seen', and is also capable of 'generalisation', namely the ability to guess correctly the output for inputs it has never seen." Farrukh Alavi, A Survey of Neural Networks: Part I, 2 IOPWE Circular (Int'l Org. of Pakistani Women Eng'rs), (July 1997), available at http://www.iopwe.org/JUL97/neural1.html.

11. "What I am going to urge here is that there is a weak and strong version of the claims for artificial intelligence in legal reasoning; that we should accept the weak version; and that we should reject the strong version, because it is based on an inadequate account of what legal reasoning is." Sunstein, supra note 5, at 4. While it is true that we can look at the fact that general systems are much more difficult to construct than expert systems to support this statement, Sunstein does not. Nor does Sunstein support this point by noting the difference between "dynamic" AI, which truly learns and generates new rules of production based on iterative experience, from "static" AI, which merely represents pre-programmed rules of production. Rather Sunstein supports this point, which is for the two aforementioned reasons tenable, with a third point, which is not; Sunstein relies on a dichotomy of deductive reasoning and analogical reasoning to explain the limitations of AI at present. In fact, the problem Sunstein has identified is either that of Mill (that inductive ampliative arguments are not necessary, only probable, as opposed to deductive arguments, which are either necessary or invalid) or that of Hume. Hume considers the logic and science amoral and implies that moral choice is arbitrary. In fact, this author disagrees with the popular representation of Hume, but that point is outside the scope of this paper.

12. Two major avenues of research have emerged over the last two decades...artificial intelligence (AI) and artificial neural networks (ANNs). While there are some similarities in both the origins and scope of both of these disciplines, there are also fundamental differences, particularly in the level of human intervention required for a working system: AI requires that all the relevant information be pre-programmed into a database, whereas ANNs can learn any required data autonomously. Consequently, AI expert systems are today used in applications where the underlying knowledge base does not significantly change with time (e.g. medical diagnostic systems), and ANNs are more suitable when the input dataset can evolve with time (e.g. real-time control systems).
Alavi, supra note 10.

13. "In 1986, Rumelhart, Hinton and Williams published the 'back-propagation' algorithm, which showed that it was possible to train a multi-layer neural architecture using a simple iterative procedure." Id.

14. See id.

15. See generally Sunstein, supra note 5.

16. See, e.g., Dan Hunter, Commercialising Legal Neural Networks, 2 J. Info. Law &Tech. (May 7, 1996), at http://elj.warwick.ac.uk/jilt/artifint/2hunter/.

17. See Alavi, supra note 10.

18. See Sunstein, supra quotation accompanying note 6.

19. See Michael Aikenhead, A Discourse on Law and Artificial Intelligence, 5 Law Tech. J.1 (June 1996), available at http://www.law.warwick.ac.uk/ltj/5-1c.html) (discussing the different theories about modeling law using AI in the law, including the use of models for analogical reasoning).

20. If a knowledge base does not have all of the necessary clauses for reasoning, ordinary hypothetical reasoning systems are unable to explain observations. In this case, it is necessary to explain such observations by abductive reasoning, supplemental reasoning, or approximate reasoning. In fact, it is somewhat difficult to find clauses to explain an observation without hints being given.
Therefore, I use an abductive strategy (CMS) to find missing clauses, and to generate plausible hypotheses to explain an observation from these clauses while referring to other clauses in the knowledge base. In this page, I show two types of inferences and combines [sic] them. One is a type of approximate inference that explains an observation using clauses that are analogous to abduced clauses without which the inference would fail. The other is a type of exact inference that explains an observation by generating clauses that are analogous to clauses in the knowledge base.
Akinori Abe, Abductive Analogical Reasoning, at http://www.kecl.ntt.co.jp/icl/about/ave/aar.html (last visited ** Nov. 25, 2002).

21. Michael Aikenhead, Legal Principles and Analogical Reasoning, Proc. Sixth Int'l Conf. Artificial Intelligence & Law: Poster Proc., 1-10 (extended abstract available at http://www.dur.ac.uk/~dla0www/centre/ic6_exab.html); Simulation in Legal Education (with Widdison R.C. and Allen T.F.W.), 5 Int'l J.L. &Info. Tech. 279-307.

22. James Popple, SHYSTER (SHYSTER is a C source code available under GPL), at http://cs.anu.edu.au/software/shyster/ (last modified Apr. 30, 1995).

23. Aikenhead acknowledges limitations in existing models but does not categorically dismiss them and, unlike Sunstein, presents critiques which will lead future work in this field. For example, Aikenhead writes:
In this context the CBR system, GREBE is particularly interesting. Branting's system addresses a major problem that arises in case based reasoning; of determining when two cases are sufficiently similar to allow analogical reasoning to occur. For two cases to be similar, their constituent elements must be sufficiently similar. How is the existence of this similarity determined? Branting's innovation is to allow GREBE to recursively use precedents to generate arguments as to why elements of cases are similar. Once the elements of cases can be said to be similar, the cases themselves can be regarded as similar. The system thus generates an argument as to why cases are similar by recursively generating an argument as to why constituent elements of cases are similar. While GREBE only operates in the domain of CBR and is not a complete argumentation system it does illustrate how useful argumentation processes can be self-referential. What is needed is an extension of this approach; applying the full theory of legal discourse. In such an approach, arguments would recursively be generated as to how and why rules or cases should apply.
Aikenhead, supra note 19.

24. Paul Brna, Imitating Analogical Reasoning, at www.comp.lancs.ac.uk/computing/research/aai-aied/people/paulb/old243prolog/section3_9.html (Jan. 9, 1996) (using Prolog to illustrate such algorithms).

25. See Alavi, supra note 10.

26. Wim Voermans et al., Introducing: AI and Law, Inst. for Language Tech. and Artificial Intelligence, at http://cwis.kub.nl/~fdl/research/ti/docs/think/3-2/intro.htm (last visited ** Nov. 25, 2002).

27. "What is legal reasoning? Let us agree that it is often analogical." Sunstein, supra note 5, at 5. In civil law jurisdictions legal reasoning is more often deductive than analogical. Therefore, there is good reason to reject Sunstein's assumption about legal reasoning.

28. "We should reject the strong version [of legal reasoning via AI] because artificial intelligence is, in principle, incapable of doing what the strong version requires (there is no way to answer that question, in principle)...." Sunstein, supra note 5, at 4. This directly contradicts a later hedge of Sunstein's when he argues that future technology may make computer AI possible. Either it is impossible and/or unverifiable or it is possible either with existing or future technology. In fact, while the speed of microprocessors and the amount of data which can be stored has changed radically in the last twenty years, processor architecutre (a bus and registers) remains basically the same as that of forty years ago (just with larger busses and more registers). Thus the technological change Sunstein awaits would, and can, occur at the software level and need not occur at the hardware level.

29. Id. at 2.

30. See Sunstein, supra quotation accompanying note 9.

31. Such a break through, while not necessary (neural networks can be programmed using existing CPUs), is possible. "As an information-processing system, the brain is to be viewed as an asynchronous massively-parallel distributed computing structure, quite distinct from the synchronous sequential von Neumann model that is the basis for most of today's CPUs." Alavi, supra note10.

32. Eric Engle, <<La convention franco américaine et la modélisation du droit fiscal par l'informatique >>,131 Fiscalite Europeenne-- Droit International Des Affaires (2003).