Access restrictions and integrity constraints clarified

Access restrictions are enforced by the DBMS facility that ensures that only authorised users gain access to the DBMS. For example, a valid user is allowed to manipulate a table with given access rights.

Integrity constraints are constraints that maintain the consistency and correctness of data.

They protect the contents of the database in totally distinct ways. When a DBMS restricts access to a user to do certain things, like granting right to only view a table rather than update it, then it is ensuring that the data providers with responsibility for maintaining the accuracy of the data content enters, updates and deletes the data while the data consumers just have the privilege to review data for their proper functioning. Also the sensitive company data pertaining to finance and personnel functions can be shielded from the prying eyes of those who have no need for direct access to this data by not authorizing access to these data areas. Essentially, access restriction is ensuring that only the data necessary and sufficient for carrying out a job is made available to the person and the rest of the data is hidden from him. Through access restrictions we can segregate responsibilities within the organisations by providing access authorization to data horizon necessary for a role. These security access restrictions are centrally defined and DBMS automatically enforces them while accessing the database (Block1,p24). The data correctness is achieved by proper responsibility sharing through access privileges and obviating the potential for unauthorised rogue data manipulations. Any SQL statement issued by a user can only be commensurate with his authorised access profile or DBMS will not execute it.

Integrity constraint ability to enforce consistent and correctness is best understood through example. If we have a geographical hierarchy with levels of company, country, region and world (eg Unilever UK, UK, Europe, Global held in COMPANIES, COUNTRIES, REGIONS and WORLD tables) then while defining a company in COMPANIES table it ensures that it is only linked to the valid countries defined in the COUNTRIES table in the database and countries and linked to the valid regions in REGIONS table. If we had linked a company to an undefined country in the database, then while aggregating regional data this rogue country would have been missed in table linkages as it is not part of the hierarchy. Also if we try to delete a region in our REGIONS table while there are countries linked to that region in the COUNTRIES table then integrity constraint could either prevent us from carrying out this operation or cascade the delete to all the records in COUNTRIES table that are linked to REGIONS table with region_id as the foreign key. All these type of rule governing referential integrity are stored in the system catalog managed by the DBMS and are automatically enforced by DBMS without required any programming intervention by the developer (Block1,p24). Although we have focussed on the referential integrity in our example, we can see the data integrity being enforced by DBMS when it enforces user-defined rules like date of birth has to be lower than date of school start or numeric phone number should not have any arithmetic operations performed on them. We can define a number of integrity constraints like email address must include ‘@’, the values for a particular column must be within 100-800 range or area code is restricted to a subset of predefined codes etc. These are all examples of integrity constraints defined to ensure the correctness and validity of the data contained in the database. The integrity constraints could also be implemented through pre-insert, pre-update, pre-delete triggers etc on the tables.

In brief, the access restriction forces correctness through security measures whilst the integrity constraints enforce correctness and consistency of contents through defined constraints on columns in the table. With access restriction no person would be able to insert, update or delete information in a table without appropriate privilege. Whilst the integrity constraint will ensure that the authorised personnel can only add data which conforms to pre-defined rules.

Why datawarehouse and OLAP tools when there is data duplication?

The data warehouse used by OLAP tools has large quantities of integrated, normally summarised, historical data which is time-stamped. The data is normally added to the data warehouse on regular frequencies rather than being updated to form an enterprise-wide, integrated repository to support data mining. All the updates to the various transactional systems are incrementally added to the data warehouse to accurately reflect the reality on the date of last extract. In the OLTP system the data has to be absolutely accurate as it is dealing with the operational transactions and it has to respond within timely fashion. Also in OLTP systems the non-current data is archived to reduce storage needs and enhance performance. The time-stamped nature of the data in the warehouse means that the business reality on a defined date can be analysed for strategic purposes without worrying about the performance impact on the transactional systems. The historical data could span a number of years to facilitate trend analysis and to seek correlations. The OLAP tools allow business users to slice and dice data, discover anomalies and drill-down to the root causes. For example, the decline in a brand’s performance could be correlated to the rise of a new launch by a competitor or the decline in advertising expenditure to support the brand or even the changing economic climate. The powerful data mining tools can carry out statistical analysis, use artificial intelligence, neural networks, and machine learning etc to unearth unexpected correlations and anomalies. There is no way such an analysis could have been done in a transactional system as it would not have access to competitor’s information or macroeconomic data. Also the normal star schema of a data warehouse is optimised for analytical processing and, may, hold aggregates. Thus the data duplication in the warehouse is being used to support a different business objective from the one expected of OLTP system. The governance structure around the warehouse ensures accuracy of data on the date of last extract from transactional systems which is incrementally added. Apart from the data extraction overhead, the OLAP system doesn’t impact the OLTP system but allows a wider business objective of data analysis to be achieved. Thus making investment in data warehouse worthwhile, despite the seemingly duplication of data.

Advantages of namespaces

Namespaces are a set of names, distinguished from other sets by being identified with a particular URI. It is a mechanism for differentiating elements. The syntax for defining a non-null prefix namespace is

xmlns:prefix:”URI”

This prefix followed by a colon is added to each tag within a vocabulary to make it unique. For example, if the prefix is ‘rdf’ then the elements would be of the form …. As xmlns is a reserve word, we can establish all the namespace definitions in a given file by searching for xmlns.

They are indispensable part of XML documents so a clear appreciation of their role is a pre-requisite for a person embarking on XML/XSL development. The key benefits offered by namespaces are:

• Facilitate use of different XML vocabularies in the same XML document by resolving conflicts stemming from identical tags being used in different vocabularies. The name clashes need to be averted. The namespace qualifier makes the tag globally unique, thus obviating any ambiguities. It is not unusual to have multiple vocabularies in a single document; for example XSLT stylesheet needs three different XML vocabularies so avoiding name clashes is of fundamental importance.

• Provides a simple, abbreviated, XML-compliant prefix for a unique uniform resource identifier (URI), thus avoiding syntactical difficulties stemming from non-compliant characters if one were forced to use full URI for qualifying tags.

• Improves readability - from a parser perspective short prefix is identical to full namespace name.

• Allows an organisation to have their distinct tags with distinct meanings through different namespaces associated with different URIs and they can all be called within same XML document., ie Adobe can have different tags for different image formats and they can all be used in a single document,

• Avoids tedium of typing by providing a mechanism for defining default namespace within a document so unqualified names automatically acquire this full qualification from parser perspective.

• To constrain a document to an XML vocabulary, we need XML Schema which can only be defined by using the reserved namespace http://www.w3.org/2001/XMLSchema which has all the attributes and elements of W3C XML schema specification. Also the document validators check document instance against the structures, elements, attributes, datatypes and constraints defined in associated XML schema. The link to associated schema in a document instance is through a namespace. Incidentally, we should appreciate that a namespace URI does not necessarily point towards anything on the implied location.

• Allows search engines to find similar documents with tags conforming to a namespace. In practice, namespace brings all the elements and attributes of a vocabulary together to be exploited by software.

Eclipse as an XML editing tool

A good XML editing tool should ideally be syntax-aware, context-sensitive, graphical and support namespaces to enhance productivity. Any experimentation with a simple text-editor like Notepad brings out all the frustration when the edited document acquires any complexity. It becomes tedious to indent the tags to improve readability, difficult to reorganise structures and a nightmare to debug errors. You may be a genius at handling XML but the fitness for purpose of a simple text-editor is undeniably questionable. Try converting a sub-element in an XML Schema instance into an attribute in a text-editor to understand the radical surgery required to achieve this simple task and the potential for errors. There are good commercial products like Stylus Studio (Progress Software Corporation) available to handle all aspects of XML development effectively but a free, feature-rich, open source product, namely Eclipse, is also admirable in its capabilities. Just look into Help\Software Update\Manage Configuration section of Eclipse to get a feel for the extraordinary variety of features provided by this integrated development environment. I will focus on three features which add value in editing XML documents and XML Schema instances.

Firstly, the syntax awareness is pervasive throughout the product. When we look at a simple xml document in Eclipse, it is immediately apparent that the elements are shown in one colour and their values in another and same is applicable to attributes. This colour scheme for handling all the colour preferences for different syntax elements is governed through the preferences management. Also if the document is linked to a schema, it automatically validates the content against the schema and underlines the text in the manner of wrong spellings in Word if the added content doesn’t adhere to the constraints of the data-type of the element. Incidentally, the contents which are not dictionary words are also underlined but in a different colour to support correction of possible mis-spelt words. The other dimension of this syntax awareness is context sensitivity.For example, when I type <>

The second major feature is the support for graphical editing. There is ‘outline’ view available which allows us to easily navigate through the tree structure by letting us expand and collapse various nodes. Clicking on any node in this outline view takes us to the corresponding element in the XML document. If this were not enough, we can toggle between design and source view in the main editing panel. In the design view, which is a refinement of outline view with content, we can easily edit the contents, whilst the source view lets us rapidly cut and paste appropriate sections. Right click on any node in design or outlook view allows us to add appropriate processing instructions, comments, elements etc.

The IDE automatically synchronises the editing place in these two distinct views to correctly reflect changes. Incidentally, the design view is displayed slightly differently graphically while editing an XML Schema instance. Carrying on the context-relevance theme, it shows sections relevant to defining a schema and their relationships. Right clicking on any of the constituents of these sections brings up a pop-up window for handling the type, ie for an element we may be able to set multiplicity while for a complex data type we may be able to add/remove elements or attributes. Also double clicking on any constituent of a section takes us to see its full definition graphically. All this has the beauty of letting us focus on one aspect type thoroughly. Once we get down to an element we can define constraints in graphic view like maximum length, enumerations etc and see the code appear automatically in the text mode. The only caution I will add is that design and source view can occasionally fall out of sync in an incomprehensible fashion. I particularly recall cutting and pasting a schema in my environment and playing with it. It insisted that a definition of an attribute from original schema were in my schema instance whereas the source view clearly showed that there was no such attribute. I believe this stems from multiple tools working together to harness the full power of XML and they occasionally don’t behave as well as one may expect.

The third impressive feature is the general XML support. We can easily click on format button to auto-indent the elements to improve readability. Or cleanup a document by automatically compressing empty element tags, or inserting required attributes, or inserting missing tags, or adding quotes around attribute values. The validate option can check a document against it schema and check it is well-formed. The refactor option allows a tag to be renamed through out a project to make it more meaningful. The schema definition template automatically refers to the appropriate URI http://www.w3.org/2001/XMLSchema. In fact, the various templates themselves are customisable. An XML document can be created from an existing schema and various options are allowed during creation. Also a catalogue of standard schemas is available. These XML support features can have occasional hiccup. I remember that a reformat of text in text-editor made a valid document against a schema invalid because the reformatting inserted a tag value on newline to make it more readable but it made it invalid against the schema as the content was from an enumeration and it didn’t like these extra whitespaces.

It should be clear that despite the small niggles the synergistic impact of these features makes Eclipse a potent environment for editing XML documents and XML Schema instances. However, it should also be noted that fans of NetBeans will notice that the similar features are available in that tool.

SOA recommendations for an SME

Here are some of the recommendations an SME should follow if it is toying with the idea of SOA and doesn't want to come to grief.

• SOA advantages of speed-to-market, reduced cost, reuse, better business-IT alignment, better and faster decision makings have to be tempered with awareness of high implementation failures, increased governance and long-term commitment (Guah,2008,pp139-40). Practitioners advise making governance a priority and ‘think big but start small’ (Sumar,2008).

• REST currently is religion for many who prophesise demise of WS-* and consider this ‘second generation’ style offers better maintenance, performance, scalability, extensibility, simplicity, security and condemn SOAP RPC as ‘DCOM for the Internet’ (Prescod,2002). SOAP uses POST to send its payload so the results cannot be cached thus thwarting scalability. If you are dealing with images then be aware that mashup with image galleries like Flickr is easier with REST (Simpkins,2009,p6). SOAP demands an image payload be packed using base64 encoding while REST allows image to be retrieved as a resource. If the business handles images and has CRUD-like interactions then Restful services commend themselves.

Although lightweight REST architectural style is deemed simpler and leverages existing HTTP protocol with intuitive resource-based URIs for web services and Amazon sees 80% REST and 20% SOAP usage (Anderson,2006), we should adhere to WS-* standards as the trading partners are likely to be using it and expect it. The standardisation, flexibility, reliability, ESB-support and ubiquitous toolkits offering productivity enhancements are compelling argument for SOAP-based services (Simpkins,2009e,p5;Haas,2005). Web services orchestration using process-modelling language, BPEL, require WSDL contract so BPEL cannot be used with Restful services (Simpkins,2009,pp5-6).

A compromise of using both approaches will not be possible for SME as supporting both styles stretches IT skill-base and investment. SOAP solution offers the benefits of large historical investment by standards bodies, tool suppliers, governments and business users and is currently more versatile so recommended despite its need for higher infrastructure investment. The WS-* standards are likely to win in the longrun (Simpkins,2009,p6).

• Protect existing investment by building adapters for legacy applications. Various wrapping approaches are available to expose the functionality (Al-Belushi and Baghdadi,2007).

• Use document/literal wrapped as the encoding model for WSDL as it is most versatile and WS-I compliant (Butek,2005).

• An Eclipse IDE with plugins, jUDDI, MySQL and Tomcat open-source environment is fit for cheap experimentation and implementation without incurring huge licensing costs of commercial products. Apache Synapse ESB can be considered. Eclipse support both SOAP and Restful services.

• Discover domain-specific services and build relationship with suppliers rather than construct everything internally. For example, banks may provide currency conversion service. Publish selected internally developed services in public UDDI to expose image catalogue and win business.

Although many other aspects can be discussed but this should give a head start with tool selection and provide a handle on rather fanatical SOAP vs REST debate.

References

Al-Belushi W. and Baghdadi Y. (2007) ‘An Approach to Wrap Legacy Applications into Web Services’, Proc. Int'l Conf. Service Systems and Service Management (ICSSSM '07), pp.1-6, June 2007.

Anderson, T. (2006) 'WS-* vs the REST', Reg Developer, 26 April [online], http://www.regdeveloper.co.uk/2006/04/29/oreilly_amazon/ (accessed 3 June 2009)

Butek, R. (2005) Which style of WSDL should I use? [online], IBM, http://www.ibm.com/developerworks/webservices/library/ws-whichwsdl/ (accessed 3 June 2009)

Ghah M. W. (2008) Managing Very Large IT Projects in Businesses and Organizations, Idea Group, Pennsylvania

Sumar S. (2008) Making Your SOA Journey Successful – Key Aspects [online], Infosys, available at http://www.infosysblogs.com/soa/2008/08/making_your_soa_journey_succes.html (accessed 3 June 2009)

Prescod, P. (2002) Second Generation Web Services [online], O'Reilly Media, Inc., http://webservices.xml.com/pub/a/ws/2002/02/06/rest.html (accessed 3 June 2009)

Simpkins, N. (2009a) ‘Block 3 part 5: Web services messaging with HTTP’, in T320 E-business Technologies: Foundations and Practice, The Open University, Milton Keynes

Is Satyam a passing phase?

The early 80s were full of articles in the trade press on the potential of Indian IT. Their key advantage in the number of engineering and mathematics graduates produced per annum was invariably cited as the propulsion engine to future greatness. The liberalisation of markets and the mantra of globalization in early 90s turned a company like Infosys, a poster-child if Indian IT, from a turnover of $5M in 1990 to $1B in 2004. The expected revenue growth of 30% per annum became the norm in Offshore IT companies. The new confident Indian operations like TCS, Wipro, Infosys etc easily met this heady growth target. There was excitement around everywhere on impending Asian century. China and India were seeing double digit growth in their economy. USA was increasingly becoming a debtor of China and people like Premji , Wipro’s Chairman, pointed towards the skills-shortages in USA market. He pointed out that Wipro is not losing its employees to its Indian competitors in American market but to the local employers who could not meet their skill requirements. People from the IT industry like Bill Gates, Eric Schmidt etc also expressed their concern on shortages of skills and they were joined in this chorus by reputable Harvard academics like Michael Porter, renowned for assessing international competitiveness. The Indian juggernaut was unstoppable. These outsources even boasted of moving up the value chain, having proved their credentials on many projects. Then Satyam happened.

Outsourcing to India’s fourth largest IT company was suppose to provide advantages of speed, agility, flexibility and focus but suddenly it has become the main preoccupation and a major distraction. The retailers like Tesco and Boots and manufacturers like Unilever and Nestle, and not to mention a host of financial institutions, are all concentrating on the finer details of their contracts and building tactical responses to mitigate risk. The Indian government is busy trying to safeguard the reputation of India Inc in the aftermath of Satyam saga, whilst the competitors like Wipro and Infosys are in over-drive to project this fiasco as an isolated event. Satyam’s $2B turnover is 4% share of the $50B market but the perceptual damage vis-à-vis governance standards is very significant indeed. When one compares it to Madoff’s $50B ponzi scheme, the scandal is relatively minor. However, the Indian prime minister has rightly recognised it as a stain on India’s corporate reputation. The confidence of IT outsourcing industry has been dented. It remains to be seen whether all the current nervousness passes eventually with minor adjustments or it proves to be a game-changing event.

Essence of eXtreme Programming

An agile development process like eXtreme Programming is a complete antithesis to the practices of waterfall model. Downplaying the importance of following plans, processes and tools, documentation whilst demanding constant receptivness to customer-induced changes comes as a culture shock. To capture the essence of the eXtreme Programming methodology I look at the four key aspects of requirements, process, architecture and risk from its perspective.

Requirements - Customer written user stories, each around three sentences, substitute formal requirement document and are used for implementation estimate during release planning meeting (Wells,2006). The actual detailed requirement is captured face-to-face with the customer during story implementation but analysis documentation is deemed unnecessary (Bennett et al, 2006,p171). It facilitates dynamically changing requirements.

Process - The adaptive, collaborative, agile and lightweight process is antithesis of traditional lifecycle methodologies (Bennett et al, 2006,p61; Wells,2006). The process is user stories driven, customer-focused, iterative and incremental where feedback is continuous. Fixing a failing process is allowed and encouraged (Sorensen,2001). The teams are small and developers work 40-hours week. Test harness has to be created prior to any coding. The code is owned collectively and all team members can change it (Bennett et al,2006,p619), The developers have to adhere to standards, design simply, have the courage to discard solutions. Constant refactoring to simplify and remove redundancy is absolutely essential. A user is a full-time team member. No specialist roles exist. There is no emphasis on using modeling tools. The code itself is deemed design documentation. Pair programming is the norm.

Architecture -  ‘you aren’t going to need it’ (YAGNI ) characterizes architecture too. Emphasis is on coding fast and relying on refactoring to fix design (Fowler,2004). The notion that we should keep things simple and not implement something till it is needed works against the idea of architecture. XP system metaphor is analogous to architecture (Herbsleb et al,2003 ). The metaphor, an evocative description, helps the team understand how the system hangs together (Herbsleb et al,2003). Simple design is emphasized and there is no upfront design (Stephens,2003). Architectural spikes or prototypes are used for testing design ideas (Sorensen,2002)

Risk - The paired programming, automated testing, continuous integration, accommodation of changing requirements, iterative development, customer involvement, improved communication, constant refactoring, coding standards, spike solutions etc are all geared towards minimizing risk (Brewer,2001;Wells,2006). The whole reason for subverting traditional practices is that XP significantly reduces risk in dynamically changing environment and these practices are at the heart of XP.

References

Bennett S., McRobb S. and Farmer R. (2006) Object-Oriented System Analysis And Design Using UML, Berkshire, McGraw-Hills

Brewer J (2001) Extreme Programming FAQ [online] Available from http://www.jera.com/techinfo/xpfaq.html 

 Fowler M. (2004) Is Design Dead? [online] Available from http://martinfowler.com/articles/designDead.html#id21325 

 Herbsleb J., Root D., and Tomayko J. (2003) The eXtreme Programming (XP) Metaphor and Software Architecture [online] Available from http://reports-archive.adm.cs.cmu.edu/anon/isri/CMU-ISRI-03-103.pdf 

 Sorensen C. F (2001) XP – eXtreme Programming: A Brief Introduction [online] Available from www.idi.ntnu.no/~carlfrs/Presentations/ExtremeProgramming.ppt 

 Stephens M. (2003) The Case Against Extreme Programming [online] Available from http://www.softwarereality.com/lifecycle/xp/safety_net.jsp 

Wells, D. (2006) Extreme Programming: A gentle introduction [online] Available from http://www.extremeprogramming.org/rules/userstories.html