CSE 636: Data Integration

Fall 2006 (#295856)

What's New

Please check the newsgroup for announcements. Lecture notes will be on this web page. A substantial part of the site will be in HTML and PDF formats. You can download Adobe Acrobat Reader here.

Staff

• Instructor: Dr. Michalis Petropoulos
• Teaching Assistant: Slawek Staworko

Newsgroup

sunyab.cse.636

Course Description

Data integration has been recognized as a research topic of big practical importance. The availability of integrated data from multiple independent, heterogenous data sources is crucial for many applications. Data integration requires combining and matching information in different sources, and resolving a variety of conflicts. XML is becoming a de facto data integration standard. With the number of data sources growing very quickly, data integration is bound to become even more important in the future. This course will survey selected theoretical and practical issues arising in data integration.

The course is based on the instructor's lectures and guest lectures. The lectures cover most of the material in the course.

Schedule

The following is a tentative schedule of lectures. Changes will appear in the "What's New" Section above.

Prerequisites

An introductory database course equivalent to CSE 562 (Database Systems)

Text

None required. A list of recommended books goes as follows:

• Database Systems: The Complete Book
  by Hector Garcia-Molina, Jeff Ullman, and Jennifer Widom
• Database Management Systems
  by Raghu Ramakrishnan and Johannes Gehrke
• Fundamentals of Database Systems
  by Ramez Elmasri, Shamkant B. Navathe
• Foundations of Databases
  by Serge Abiteboul, Richard Hull, Victor Vianu
• Data on the Web
  By Abiteboul, Buneman and Suciu

Grade Computation

• Assignments: 10% (set of 2, 5% each)
• Midterm: 20%
• Final: 20%
• Project: 45%
• Participation: 5%
• Grades

Practice Problems

• Assignment 2 Solutions
• SchemaSQL Practice Problem

Project

• Phase 1 Specification
  • Example web application: myusername.war
  • README.txt
  • Apache Ant
  • Apache Tomcat
  • An explanation on "Query String" from Wikipedia
    Note: They refer to servlets as CGI programs
• Phase 2 Specification
  • Practice problem on GAV-style integration
    • Source A: Schema - Data
    • Source B: Schema - Data
    • Integrated Schema
    • Attempt 1: XQuery - Data Union, Restructuring
    • Attempt 2: XQuery - Data Duplicate Elimination, AntiSemiJoin
    • Attempt 3: XQuery - Data Nested AntiSemiJoin
  • Project Phase 2 Test Cases.pdf
• Phase 3 Specification
  • BEA WebLogic Workshop 8.1 with SP6 (DB Server Copy)
  • BEA AquaLogic Data Services Platform 2.5 (DB Server Copy)
  • BEA AquaLogic Data Services Platform Documentation
  • BEA AquaLogic Data Services Platform Tutorial 1
  • BEA AquaLogic Data Services Platform Tutorial 2

Resources

• XML 1.0 W3C Recommendation
  Includes DTDs.
• Document Object Model (DOM) W3C Recommendation
• XML Tutorials ( ZVON, W3Schools )
• DTD Tutorials ( ZVON, W3Schools )
• Online DTD Validator

• XML Schema W3C Recommendation
  Consists of three specifications. Start with XML Schema Part 0: Primer.
• XML Schemas: Best Practices
• XML Schema Tutorials ( ZVON, W3Schools )
• Apache Xerces
  XML Parser, XML Schema Parser and XML Schema Validator. Implementation of both DOM & SAX.
  In order to validate your XML Schemas and your XML files against them, follow these steps:
  • Make sure you have Java JDK 5.0 installed on your computer. If not, download the latest version from here.
  • Download the latest binary distribution of Apache Xerces for Java from here: .zip | .gz
  • Extract all files from the Apache Xerces .zip or .gz file on your computer.
  • From the command line, change the current directory to where you extracted Apache Xerces.
  • Execute the following command:
    
    On Windows
    java -cp xercesImpl.jar;xercesSamples.jar dom.Counter -v -s <path to your XML file>
    Example: java -cp xercesImpl.jar;xercesSamples.jar dom.Counter -v -s ..\..\project\data.xml
    
    On Linux
    java -cp xercesImpl.jar:xercesSamples.jar dom.Counter -v -s <path to your XML file>
    Example: java -cp xercesImpl.jar:xercesSamples.jar dom.Counter -v -s ../../project/data.xml
    
    
  • The above command will validate your XML file against the XML Schema specified and will print your XML file in standard output.
  • Here is an XML Schema and an XML file you can use to test the above command.
  • In the example XML file, note how you specify the XML Schema that it conforms to.
• Another Online XML Schema Validator

• XML Query (XQuery) W3C Candidate Recommendation
  Consists of multiple specifications. Includes XPath. Start with XQuery 1.0: An XML Query Language.
• XPath Tutorials ( ZVON, W3Schools )
• XQuery Tutorials ( IPEDO, W3Schools )
• IPSI-XQ XQuery
  Java implementation of the W3C XQuery standard. Includes a nice and simple GUI for running your XQuery expressions.
• Saxon
  Open Source XQuery processor developed by Michael Kay.

• Web Services Description Language (WSDL) 1.1 W3C Note
• W3C Web Services Activity
  Consists of multiple groups. Includes WSDL and SOAP, among other specifications.
• Web Services Tutorials ( BRICS, W3Schools WSDL, W3Schools Web Services )
• A Busy Developers Guide to WSDL 1.1
• Apache Axis
  Java implementation of SOAP.
• Apache Axis Tutorial (PDF)
• Apache Tomcat
  This is the J2EE-compliant application server you will use for the project.
• The J2EE 1.4 Tutorial
  This tutorial covers many web technologies, including web application configuration and Java servlets.
• The Java Web Services Tutorial
  This tutorial covers many web services-related technologies.
• XMethods
  A collection of information about existing Web services.
• XMethods' WSDL Validator
  Make sure your WSDL makes sense. Your WSDL must be accessible over the web, i.e., you must provide a WSDL URL.

• Datalog Educational System (DES) is a basic implementation of Datalog with recursion and stratified negation.

Rules & Policies

Exam Rules

Group study and discussion are encouraged, but midterms and finals must be your own work. For coding assignments, if you use a piece of code which you borrowed from elsewhere and therefore did not write yourself, make sure you comment it to show this. Zero tolerance on plagiarism/cheating: consult the University Code of Conduct for details on consequences of academic misconduct, and see also the academic integrity policy of the CSE department.

Midterms & Finals

Midterms and finals are due at the beginning of the lecture of the due date. Late midterms and finals will not be accepted.

Grading Policies

Write clear arguments. Be neat and precise. Getting the right answer may not be enough. The derivation and quality of writing counts! Don't write many different things in hope that you'll get the points if one of them is the right one. Indeed, you will lose points if you follow such a policy.