Packages

  • package root
    Definition Classes
    root
  • package edu
    Definition Classes
    root
  • package chop
    Definition Classes
    edu
  • package cbmi
    Definition Classes
    chop
  • package dataExpress
    Definition Classes
    cbmi
  • package backends

    Provides database wrappers and connectivity for MySQL, Postgres, Oracle, and SQLite JDBC drivers.

    Provides database wrappers and connectivity for MySQL, Postgres, Oracle, and SQLite JDBC drivers.

    Overview

    The support for individual databases is broken out into several classes:

    The SqlBackend for the corresponding database is the primary means of establishing a datsbase connection when using DataExpress. However, It is important to note that it is generally easier, and safer to use the edu.chop.cbmi.dataExpress.backends.SqlBackendFactory to establish the necessary backend and set up the connection

    In general the SQLDialect and BackendProvider are used as part of a SQLBackend, and not directly.

    Usage

    The edu.chop.cbmi.dataExpress.backends.SqlBackend requires properties to fully configure itself. Typically these are loaded from a file, but can also be supplied through a standard java.util.Properties object. The SQL backend has very simple requirements for properties files. They must contain the following keys:

    • driverClassName: The class name for the actual database driver
    • jdbcUri: This is the URI required by the database driver. It is passed through directly to JDBC so you are free to specify schemas and other information here
    • username: The username to authenticate against the database (not required for non-authenticated RDBMS systems such as SQLite) -password: The password for accessing the database

    DataExpress will pass along any other properties it finds directly to JDBC, so in cases where a database requires specific connection configuration (e.g. to enable SSL for Postgres), those settings can be passed along.

    In some cases, DataExpress will set specific properties on the connection to ensure predictable behavior. Most of these have been empirically determined by our testing given the narrow use cases for DataExpress. Where possible, we have made it possible for you to override these defaults in your own configuration file, however this is generally not recommended.

    Definition Classes
    dataExpress
    Example: 

    1. scala> import edu.chop.cbmi.dataExpress.backends._
import edu.chop.cbmi.dataExpress.backends._

scala> val backend = SqlBackendFactory "src/test/resources/sqlite_test.properties"
backend: edu.chop.cbmi.dataExpress.backends.SqlBackend = SqlBackend({jdbcUri=jdbc:sqlite:target/source.sqlite},SqLiteDialect,org.sqlite.JDBC)

scala> backend connect
res0: java.sql.Connection = org.sqlite.Conn@7168f171

scala> backend execute """CREATE TABLE de_test(id INTEGER, first_name VARCHAR, last_name VARCHAR)"""
res1: Boolean = false

scala> backend commit
res3: Boolean = false

scala> backend executeQuery "select count(*) from de_test"
res4: java.sql.ResultSet = org.sqlite.RS@3b070e76

scala> res4.getInt(1)
res5: Int = 0
    Note

    The SQL backends return jdbc ResultSets which are typically a little more "low-level" than what you would probably want for ETL purporses. The edu.chop.cbmi.dataExpress.dataModels.sql.SqlRelation is a much more convenient object to use for ETL.

  • package dataModels
    Definition Classes
    dataExpress
  • package file
  • package sql
  • ColumnNameGenerator
  • DataRow
  • DataTable
  • DataType
  • RichOption
  • SeqColumnNames
  • SimpleDataTable
  • package dataWriters
    Definition Classes
    dataExpress
  • package dsl
    Definition Classes
    dataExpress
  • package exceptions
    Definition Classes
    dataExpress
  • package logging
    Definition Classes
    dataExpress

package dataModels

Ordering
  1. Alphabetic
Visibility
  1. Public
  2. All

Type Members

  1. trait ColumnNameGenerator extends AnyRef

    When mapping column names to methods via Scala's dynamic trait, it is necessary to generate names that comply with Scala's rules.

    When mapping column names to methods via Scala's dynamic trait, it is necessary to generate names that comply with Scala's rules. A name generator is needed to ensure the method names are properly formed.

  2. case class DataRow[+T](column_names: Seq[String])(data: Seq[Option[T]]) extends Seq[Option[T]] with Dynamic with Product with Serializable

    A Wrapper class that extends Seq[T] with scala.Dynamic to enable row element access using the column names with dot notation e.g.

    A Wrapper class that extends Seq[T] with scala.Dynamic to enable row element access using the column names with dot notation e.g. Given a DataRow instance, dr, with column_names = Seq("a","b") and data = Seq(1,2), the values can be accessed as dr.a and dr.b

  3. abstract class DataTable[+T] extends Iterator[DataRow[T]] with Dynamic

    base class for other data representation classes organized as a 2-D table with column names.

  4. abstract class DataType extends AnyRef
  5. class RichOption[T] extends AnyRef
  6. case class SeqColumnNames(column_names: Seq[String]) extends ColumnNameGenerator with Product with Serializable
  7. case class SimpleDataTable[+T] extends DataTable[T] with Iterator[DataRow[T]] with Product with Serializable

    A simple immutable edu.chop.cbmi.dataExpress.dataModels.DataTable that maintains all data elements in memory

Value Members

  1. object DataRow extends Serializable

    Factory for edu.chop.cbmi.dataExpress.dataModels.DataRow

  2. object DataTable

    Factory for edu.chop.cbmi.dataExpress.dataModels.DataTable

  3. object RichOption

Ungrouped