{title Making an Adaptive KB}
{subtitle Work}
{author Chris Dent cdent@burningchrome.com}
{docid kb.1}
{date April 5, 2004}

It's been in the works for many months. It seems the time is finally here. Active discussion on the next iteration of [http://www.iu.edu Indiana University's] [http://kb.indiana.edu Knowledge Base] has begun. {nid 3TT}

This comes at an interesting time in systems development: an awareness of the desirability of unintended uses of systems and reuse of code is growing alongside a growing understanding of the importance of systems that are well suited to the many dimensions of the environment in which they are situated. {nid 3TU}

[http://www.shirky.com/writings/situated_software.html Situated software], a term recently popularized by [http://www.shirky.com/ Clay Shirky], refers to software that is built quickly in response to the needs or desires of a relatively small social group. Situated software "isn't a technological strategy so much as an attitude about closeness of fit between software and its group of users, and a refusal to embrace scale, generality or completeness as unqualified virtues." {nid 3TV}

This attitude exists in stark opposition to traditional views of software development. The perceived expense of the initial development process and the even higher perceived expense of making adjustments after release has made scalability and control along many dimensions (number of users, time, rate of use) a very high priority. {nid 3TW}

Proponents of agile methodologies have (accurately) claimed that such scalability requires perfect knowledge of requirements. Given the only reliable rule about the real world is [http://home.att.net/~Flexibility/Counter/Intuitive.htm things change] true scalability is impossible. {nid 3TX}

Agile developers, understanding the challenge of constant change, have responded with a [http://agilemanifesto.org/principles.html manifesto] predicated on acceptance of change and the necessity of constant refactoring: in system requirements, system tools, and the social systems that surround the development process. {nid 3TY}

Enabling unintended or unexpected uses of systems is one of a host of ways to deal with change. If a system is pre-built for flexibility it is more adaptable to ever changing needs. {nid 3TZ}

The success of the world wide web can be traced back to one fundamental principle: openness. The web is "[http://www.smallpieces.com/content/preface.html many small pieces loosely joined]". {nid 3U0}

It is no coincidence that web services and other service oriented architectures have gained in popularity in parallel with agile methodologies. Openness, loose-coupling, and smallness are the enabling techniques of agile technology. {nid 3U1}

Openness, loose-coupling and smallness are also the enabling technologies of situated software. Therein lies the key to building an adaptive KB. {nid 3U2}

Situated software systems, as Shirky says, are purpose built for a social group. Those systems, though, are not built from scratch. They are built using popularly available tools and technologies that are open (using HTTP and other open standards, open source databases and languages) and fairly mature. They provide stable building blocks on which to loosely-couple new, small pieces. These ideas are already embedded in the service oriented design of the [http://sakaiproject.org/ Sakai Project] so a Sakai oriented Knowledge Base project is well positioned. {nid 3U3}

The current iteration of the Knowledge Base (KB3) was developed, in part, out of a desire to more effectively respond to change. An extensive application program interface was developed in an attempt to separate the application front end from the minutiae of data handling. The result was certainly an improvement over previous systems, but today's system remains a single entity of tightly coupled pieces wherein change has many cascading effects and the many different user groups are effectively considered one. {nid 3U4}

A Knowledge Base system can be created that is made up of many small pieces. Those pieces can be loosely joined to create not just one but many instances of situated software that respond to the needs of multiple, evolving, user populations. {nid 3U5}

A small pieces Knowledge Base is one in which the many functions of a knowledge base are pulled away from a single central service model, to a model using multiple service providers accessed by multiple, small applications. The requisite pieces are discovered by finding, solidifying and making gateways in the boundaries between functions in the KB. {nid 3U6}

As an example, the small pieces model has already been demonstrated in the creation of a SOAP interface to the presentation layer of the Knowledge Base. By itself, using SOAP to access the KB provides only one (albeit valuable) thing: remote access. What makes the interface more valuable is that it defines only three methods of passing through the boundary between the KB data repository and the presentation system provided by the SOAP client. That narrow passage makes the interface simple and small and thus usable in a straightforward manner by various social groups. The SOAP interface has already been used to create tools for the Sakai Project, IUWare Online, the PIE time-tracking system, the Hermes notification system and an extremely small, fully-operational, remote, command-line interface to the public KB. {nid 3U7}

By dividing the KB into pieces with solid boundaries, each piece can be developed and tested independently of other pieces. Each piece can be exposed to any other piece, other applications, other services and other environments not yet imagined. Code from other service oriented projects can be reused more easily and services within the KB can be reused elsewhere. {nid 3U8}

There are obvious large chunks in the KB which can be used as starting points for determining the pieces: {nid 3U9}

* data repository {nid 3UA}
* content editing {nid 3UB}
* content presentation {nid 3UC}
* indexing {nid 3UD}
* reporting {nid 3UE}

This list is incomplete and misleading and should not be used a sole guide. It is incomplete in that each section can be broken down further into still smaller pieces. For example, content presentation can be subdivided into generation, retrieval, and presentation, with perhaps further subdivision within. It is misleading in that there are obvious interconnections between each of the pieces. To succeed, a loosely-coupled model must enforce the gateways between pieces and be predicated on ignorance of the internal workings of the services that represent each piece, much like encapsulation of objects. {nid 3UF}

Gateways between chunks should, like entities in a well designed database, model real world transactions rather than being patterned on the constraints of the artifact world that is created in the system. A gateway into the content editing system should define setDocument() rather than a series of smaller methods. The former reflects the desire of the human, situated outside the system who is performing the task, while the latter reflects the tasks of the system within. {nid 3UG}

Using small pieces breaks development into manageable chunks. Any development project [http://www.dehora.net/journal/archives/000423.html is no simple task] but when there are clearly delineated pieces, complexity can be shifted and bounded. Tasks are easier to identify and the answer to the inevitable question of "what should I do now?" is more apparent. Test driven development is easier to achiever with smaller pieces. The unfortunate reality of developer turnover and lack of developer experience is less of a problem with small pieces as they are easier to grasp. {nid 3UH}

A small pieces Knowledge Base will not and cannot foresee all eventualities but it can be more prepared for them by being adaptable. That adaptability can be achieved by breaking the monolithic service of the existing KB into smaller pieces that act independently but also interoperate. Additional small pieces of situated software acting as front ends to the KB system can meet the needs of emerging groups in an ad-hoc fashion. {nid 3UI}

The ethos that drives the model described above is informed by the references in the text. These documents are relevant to the task at hand. Here they are again for easy reference: {nid 3UJ}

* [https://kb-dev.indiana.edu/arts-public/1076390738.5685.wiki Idioms and Standards in Knowledge Base Reuse]. Chris Dent. An early version of similar thoughts related to the SOAP interface to the KB. {nid 3UK}
* [http://www.burningchrome.com:8000/~cdent/arts/my/1.1.wiki The Computer As Tool: From Interaction to Augmentation]. Chris Dent. Task and craft oriented approach to system design. {nid 3UL}
* [http://www.jwz.org/doc/worse-is-better.html The Rise of "Worse if Better"]. Richard Gabriel. Done is better than perfect. {nid 3UM}
* [http://www.heyotwell.com/heyblog/archives/000305.html Structure & Situated Software]. heyblog. Commentary on Shirky's situated software thoughts with reference to Alexandrian patterns and the inefficiencies of rigid and/or tree-oriented hierarchies in systems. {nid 3UN}
* [http://www.dehora.net/journal/archives/000423.html Better is better: improving productivity through programming languages]. Bill de Hora. Development is difficult, literate, skillful and always will be. {nid 3UO}
* [http://home.att.net/~Flexibility/Counter/Intuitive.htm Counter Intuitive Management of Information Technology]. Bruce Johnson, Walt Woolfolk, Peter Ligezinski. Destroying some IT management myths to allow and support change. {nid 3UP}
* [http://www.groove.net/blog/?month=03&year=2004#3EB75869-B4E2-4EB4-B272-E17D3B3D6882 Adaptive Distributed Computing]. Ray Ozzie.  {nid 3UQ}
* [http://www.smallpieces.com/content/preface.html Preface of Small Pieces Loosely Joined]. David Weinberger. {nid 3UR}
* [http://www.shirky.com/writings/situated_software.html Situated Software]. Clay Shirky. {nid 3US}
* [http://agilemanifesto.org/principles.html Agile Manifesto]. Various. Manifesto from the leaders of the agile methodologies movement. {nid 3UT}