database « WordPress.com Tag Feed

Designing a Database ER Model with Dia

vailo — Mon, 07 Jul 2008 07:35:28 +0000

When it comes to web development many people seem to forget one important thing. They all start with designing the graphics and page layouts and making sure the code follows every standard and conventions there is, but sadly that's it. Many don't think of database design as a important piece in the development. When you work on your code and you are connected to a MySQL database, how many times have you been forced to edit the database schema? If you answer is: many times. Boy, we have some work to do. The problem with designing a database might not occur when you are working alone or in pair but as soon as you start out with a three member team or larger a database design is highly recommended.

What do I mean with designing a database? First of all we need to figure out what makes a database. If we take a look at a database, any database, we can easily divide it into three layers:

The conceptual layer
The implementation layer
The physical layer

The conceptual layer is where we design. This is where we design the database schema with the tables, constraints and relationships. The design we create here can be transferred to any database because it's not connected to a specific database type. The conceptual layer is an easy to understand explanations of the database. This is where we are going to look deeper today.

The implementation layer is where we actually implement our conceptual database schema. This is where you choose which type of database we want to work with, for example a relational database such as MySQL. Since the conceptual layer is just an explanation of a database of any type we can use the conceptual design in many different implementation layers.

Lastly the physical layer. This is where we actually store our database and data on the hard drive. Here we setup how we want to access files, index and more.

OK, we go back to the conceptual layer and we are going to introduce a program to use when we want to design a database conceptual schema. The program is called Dia and is available for free.

If you are totally new to the concept of relational database and have absolutely no clue about what I went through earlier I will try to explain a few things before start with designing a test database. When we design we will look upon three different types of objects.

A relation is the same as a table, but we call it a relation. When we design we are looking for any type of real world object like a car, books or users. A relation is displayed as a rectangular.
A relationship is the constraint between two relations. A car must be owned by a user for example. A relationship is displayed as a diamond shape.
A attribute. A relation has attributes and as we might know it different fields. A user might have id, username, password and email attribute. These are connected to a relation via a thin line and the attribute itself is surrounded by a circle.

So now we have three different objects we can use and it looks fairly easy now doesn't it? We basically have all the tools we need to create a conceptual design everyone can look at. As I mentioned the important thing about designing a database is that it's crucial not to change the database schema. Of course you might have to do it during development and in rare cases when the site is online but as a ground rule you shouldn't change the database schema due to the fact that many other applications and users might depend on it.

OK, when you have downloaded the Dia program, install it and run it. On startup two different windows will popup: a workbench and a menu box. In the menu box there is a drop down menu with the default value 'Diverse'. Click on that and on the second link from the bottom there is a little arrow pointing to the right, click on that and choose the 'ER'. This stands for Entity-Relationship module. Why do we say Entity-Relationship module when we just talked about Relation, Relationships and Attributes? A relation is sometimes also called a Entity. What we changed in the menu is that now the Dia program is in the ER mode. Under the drop down menu we just entered a few new icons have popped up.

E inside of a rectangular: this is our relation/entity button
E inside of a double rectangular: this is also our relation/entity button but for weak entities.
R in a diamond: our relationship button
A in a circle: our attribute button

Simple isn't it? Now we can start designing our conceptual database schema. We will start with a simple example. Lets say we want to save users in our database. Each user has a unique id, username, password, email and gender. How can we display this? We start with dragging out a Entity box. Double click on it to access the settings. Name it to 'User' and click apply. The entity is done. Now we should add the attributes. Click on the circle with the A inside. First we create the id attribute, make sure to check the 'key' option in the attribute settings panel. When you press apply the attribute will be underlined, this shows that the attribute is a key. Continue adding the username, password, email and gender attribute but don't apply the key option. Finally draw a line between each attribute and connect it to the entity box. I used the line tool and not the double lined button found under the Entity button. Make sure you don't use arrows, just a straight line. We should come up with something like this:

[wp_caption id="attachment_127" align="alignnone" width="262" caption="Dia example 1"][/wp_caption]

Next we add a new entity Car. It will have two attributes: a registration number and color. The registration number will be the unique attribute for a car and the color attribute can be multivalued. Even though the normalization rules tells us that multivalued fields in a database is wrong, we need to remember that we are working on the conceptual design. We do not think of terms like implementation or MySQL.

We add a new Entity and name it Car. We add the registration number attribute and add the 'key' option. For the color attribute we make it multivalued by simple adding that option key. The attribute will be boxed in by a double lined circle, this is telling us that the attribute is multivalued. We take our design one step further. Each User is able to own several cars but one Car can only be owned by one User. How do we display this? We use the Relationship object! Insert a Relationship object between our User and Car entity. Name it owns. Double click to access the settings panel in the left cordiality enter 'n' and in the right enter 1. Finally connect each Entity to the Relationship. It will look something like this:

[wp_caption id="attachment_128" align="alignnone" width="300" caption="Dia example 2"][/wp_caption]

Now we have done something really cool. We have created a design of a database with two entities and one relationship between them. It's easy to see the attributes of each entity and the relationships between them. Imaging a very big database, a design like this would certainly ease it up a bit!

This is only a short guide on how to use the ER model. All I wanted to do was to introduce the Dia program and how you can use it. You will find a useful article about the ER model at the link below. OK, that's it. A very short introduction on the ER-modelling and how you can use Dia to create it. Thanks for your time!

Wikipedias description of the ER Modelling

SQLAuthority News - Thank You to Awarding Author SQL MVP

pinaldave — Mon, 07 Jul 2008 01:30:17 +0000

I received award from Microsoft for SQL Server Most Valuable Professional a week ago. I have received many many congratulations messages from many readers for getting this award. I thank all of you for sending me messages and your wishes.

Honestly, I think this is all of yours award and I am just receiving this award for everybody who is reading and participating on this community forum. My goal is that more and more user participation occurs on this website and I publish few articles which are really contribution from readers.

If you are reading this blog and have any idea which you think can be helpful to other readers, please send it to me at my email address and I will publish article with your name on this blog. I will be happy to have readers contribution. If you want me to review book or speak in your conference please contat me as well at my email address.

Once again, Thank YOU, without all of success of this site is not possible.

Pinal Dave (http://www.SQLAuthority.com)

The PHP Series Examples

vailo — Sun, 06 Jul 2008 08:03:07 +0000

Reading a description or tutorial is always interesting but it can be hard to understand everything and all the features included. This post will use our 6 PHP classes we have created and look upon a few example on how they can interact with each other.

We will begin with a example between the Database and Paging class. We start by including the two classes in our default.php page using 'require'. We create objects of our classes and we simply start with a connection to our database and starts our paging.

$data = new Database('database', 'server', 'user', 'password'); $paging = new Paging('SELECT * FROM person', 5, 'result' ,$data);

This is what we need to do to instantiate our classes. Next up is to loop through the resultset the paging object gets. In this example we will use the public fetchObject() method in the Paging class.

while($rs = $paging->fetchObject()): echo $rs->username . '<br/>'; endwhile;

To finish off this example we need to echo out the paging links.

echo $paging->getLinks();

Simple! We are done! To get the example working you need to make sure database, server, user and password are correct to access your MySQL database. The SQL string needs to be replaced by a real SQL string and lastly, the $rs->username must be change to a column name in your resultset.

Download the Database and Paging example

Next up a Database and Record class example. We start again by creating objects of our Database and Record class:

$data = new Database('database', 'server', 'user', 'password'); $record = new Record('person', 'id', $data);

After that we go straight to save a new record in the person table. In the example I know that there are two fields in the person table: username and password. Since we didn't include fields in the instantiating of the Record class all fields in the person table is open to us.

$record->username = 'Niklas'; $record->password = 'test';

And lastly we save the record using the save() method making a new record entry in the person table.

$record->save();

We continue with the example and this time we are going to try to find an entry using the find() method and update the password. We are going search for the latest inserted user. Since we just added a new user to the person table we will access that tuple and update the password value.

$record = new Record('person', 'id', $data); $id = $data->fetchTuple('SELECT id FROM person ORDER BY id DESC'); $record->find($id);

We be sure we have a record to work with we echo out both the username and the password.

echo $record->username; echo $record->password;

The result will be 'Niklas' and 'test'. OK, now we want to update the password from 'test' to 'updated'. Lastly we save the record and echo out the password to see if the task is complete.

$record->password = 'updated'; $record->save(); echo $record->password;

The answer: 'updated'. Perfect! To be sure that the value is saved in the database you need to check the table using your MySQL Query Browser or simple create a new Record object and find the id we just updated. Now we have code for saving a new record and updating an existing one.

Download the Database and Record example

We continue with another example between the Database and Forms class. We will first create a form from scratch and then try to auto generate one using the generate() method in the Forms class. The last task will be a bit hard to just download and get up and running smoothly since it's tightly integrated with my personal MySQL database. Anyway, you will see the idea on how to use the classes.

We start as we did before with creating objects from our classes:

$data = new Database('database', 'server', 'user', 'password'); $form = new Form('GET', 'example_database_forms.php');

We use the same Database settings as before and the Form object will use 'GET' as the method and go to the example_database_forms.php page on submit. We will create two fields, a username field, password field and then include a submit button.

$form->addField('username', 'text', array('required'=>true, 'format'=>'letter', 'value'=>'Username', 'min'=>3)); $form->addField('password', 'password', array('required'=>true, 'format'=>'string', 'value'=>'Password')); $form->addField('submit', 'submit', array('value'=>'Submit')); echo $form->create();

Both fields are required, we have set a format pattern, letter and string, and the Username field got a min validation rule. Both fields also got a default value and the labels are showing. We have more validation and styling rules to use on our Form:

label: set the label for each field.
showLabel: true or false, decides if the field label should be visible or not.
boxClass: each field is surrounded by a div box, this sets the class of that box.
fieldClass: each field has a class, this sets the class of the field.
snippet: a code snippet which is placed at the very end of the field.

To include one more validation rule you simply include a key => value into the last parameter, the array. This example requires not only the Database and Forms class to be included, the Common class is used by the Forms class and must be included too. We cannot forget about the validation JavaScript either so in the head we insert that.

Now we have created a small form from scratch and now we will auto-generate a form based on a MySQL table. To do this we need to provide a database object and a table name when we create our Forms object.

$form = new Form('POST', 'example_database_forms.php', array('db'=>$data, 'table'=>'person'));

The auto-generation of the form is simple. The next line does it all for us.

$form->generate();

Now we have the whole table as a form in our Form object. We can't see anything on the screen because we haven't created the HTML tags yet. In the example we are not using any default values and rules either. The next example is taken care of that. But, back to this example first. All we need to do now is to include a submit button and echo out the form.

$form->addField('submit', 'submit', array('value'=>'Submit')); echo $form->create();

These four lines of code have generated a form from a table. You notice will notice a little * in the right corner of the labels, those are required. The generate() method takes care of a lot of cool things.

We stick to the Forms class example for a little while longer. Now we will try to fill an auto-generated Form with default value. Say we want to edit a user in our database. We will also say that each field requires to be longer than 3 chars. To complete this task there are a few things we must look at.

First include the Record class so we will be able to get the user values.
Give the generate() method a parameter.

So, we still start with including the Record class and then the instantiating of the objects:

require'(lib/Record.php'); $record = new Record('person', 'id', $data); $form = new Form('POST', 'example_database_forms.php', array('db'=>$data, 'table'=>'person'));

Next we need to fetch the values from a $record object. We start with a search to find the user we are looking for and then fetching the values from that user.

$id = $data->fetchTuple('SELECT id FROM person ORDER BY id DESC'); $record->find($id); $values = $record->getValues();

OK, we search for the last inserted user id and used the find() method in the Record class and saved the values from that class into a local variable $values. All we have to do now is to include those values in the generate() method parameter. We also said that each field must be at least 3 chars long, this is also included in the generate() parameter.

$form->generate(array('values'=>$values, 'min'=>3));

Simple. Now if you look at your result, and everything is OK, you will see that your form has default values and when submitting is check so each field is longer that 3 chars. We end it all with adding a submit button and echo the form out.

$form->addField('submit', 'submit', array('value'=>'Submit')); echo $form->create();

[wp_caption id="attachment_124" align="aligncenter" width="300" caption="Database and Forms example"][/wp_caption]

Download the Database and Forms example

The final example will be between the Database, Forms and Record classes again. We will take a look how you fast and easy can save a whole form using just a few lines. Our header will look like this:

require('lib/Database.php'); require('lib/Common.php'); require('lib/Form.php'); require('lib/Record.php'); session_start();

We have included everything we need and we start off by creating our Database object:

$data = new Database('database', 'server', 'user', 'password');

Nothing new there, the Database object has been the same throughout all these examples. Next we do a check if a $_GET['create'] has been provided. If it is, we are going to save the incoming information otherwise we echo out our form. Now, you shouldn't use 'GET' when you are posting things, especially not registration forms. This is just a example so we are able to see which information we are sending. Do NOT use 'GET' in real applications.

The Form object will not look that different from before. We have added a hidden field named 'create' to the form just so we can catch that in our if / else statement.

if(isset($_GET['create'])) { $record = new Record('person', 'id', $data); $record->set($_GET); $record->save(); echo 'User is saved!'; } else { $form = new Form('GET', 'example_database_forms_record.php'); $form->addField('username', 'text', array('required'=>true, 'format'=>'letter', 'min'=>3)); $form->addField('password', 'password', array('required'=>true, 'format'=>'string')); $form->addField('create', 'hidden', array('value'=>true)); $form->addField('submit', 'submit', array('value'=>'Submit')); echo $form->create(); }

[wp_caption id="attachment_125" align="aligncenter" width="300" caption="Database, Form and Record example"][/wp_caption]

So there you have it! A few examples on how to use the classes we have been working on! Everyone is available for download so you can check them out more carefully. Just remember to change the parameters so they will work in your machine and towards your MySQL database. Enjoy!

Download Oracle 9i (3 CD Set up)

tarunreflex — Sun, 06 Jul 2008 07:49:28 +0000

Oracle is a leading provide of database technology to support business intelligence. For any company using their database to support business intelligence or data warehouse applications.

- Automated ETL Capabilities Simplifies the Construction and Development of Business Intelligence Solutions and Reduces Their Cost
- Provides Limitless Scalability and Unmatched Performance and Availability For Business Intelligence Applications Without Costly Application or Data Reconfiguration
- Integrated Business Intelligence Infrastructure Speeds Time to Market For Business Intelligence Applications, Optimizes Their Performance, and Simplifies Their Change
- Expands The Role Of The Oracle Database To Become The Platform For Analytical Applications For The Internet.
- Simplifies the Construction, Management, and Maintenance of High Availability Business Intelligence Applications
- Delivers More Value Per Click Through Real-Time Mining and Recommendations For e-Business Customers
- SQL Enhancements / New Features Streamline Business Intelligence Application Development By Facilitating Integration, Compatibility, and Portability With Popular RDBMS’s and Existing ETL Environments
- Reduces Application Specific Security Development and Cost By Integrating Core Security Functionality Into the Database to Be Leveraged Across All Business Intelligence Applications
- Provides Utilities and Wizards to Accelerate and Ease the Migration of Any Business Intelligence Application to Oracle 9i

Building your own Photo Archive

pfauland — Sat, 05 Jul 2008 18:51:39 +0000

Murphy's Law ... Or Why NOW is the right moment

for buliding your own photo archive.

When did you check the air pressure of your tires last time ? Be honest, you don't even remember ! When did you visit the gas station last time ? Before
the car runs out of fuel of course. Some things only come to our mind with a big bang when it's too late. And believe me, a flat tire is annoying but still something that
can get fixed and after an hour or two you are up and running again.
Now, think about your photo archive for a moment. Is it really safe ? "Of course, I have my important files ... mmmh ... Probably most of them on DVDs ..." I
don't want to scare anybody, but it's time to think about some changes. The possibility of loosing the photographs of the honey-moon trip to Hawaii, the first steps of the daughter, or
loosing a client - who will not recommend you any further - after you had to tell him, that the photographs you took for him are .... somehow not available any more.

As data storage and archiving is so essential for everybody maintaining a large photo archive, I will to go a bit more
into detail on the Hard- and Software side of the business.

What do we need - Hardware wise ? This depends on three factors :

The *importance* of your data (Think for a moment on the consequences in case you
would lose files).

The *amount* of data (Are we talking about 200 MB, 5 GB or 500 GB ?).

And last but not least the *organization* of your data (Does a "file card system" do the job ? You need a database ?
With only the file names ? Or with meta-tags, thumbnails and keywords ? You handle everything form your laptop or do other people / colleagues / friends need access via the web?)

As usual, money plays also an important role. Professional storage and archiving solutions can cost thousands of Dollars, Euros, etc. But in most cases a decent investment will
bring the maximum amount of safety that will make you sleep much better.

READ THE FULL ARTICLE HERE ... !

CAD Manager, Engineering and Mechanical, Aldermaston

mrcas — Fri, 04 Jul 2008 19:50:10 +0000

CAD Manager, Engineering and Mechanical

Reference: 1215082349

ERS have a Client who is looking for a CAD Manager for development of application media for the knowledge management of company configuration records.

Location

Aldermaston

Salary

Competitive salary based on qualifications and experience

Job Responsibilities

A working understanding of relevant British and European Standards and legislative requirements.

Compliance of drawings with company standards.

Maintenance of site drawing standards database.

Provision of internal drawing capabilities to all facilities and projects.

Development of an integrated configuration team, looking at application and storage of configuration data.

Completion of drawings to meet programme requirements.

Carry out surveys to support the completion of drawings.

Preparation of work to pass onto draughtsmen for production of drawings.

Verification of the drawings to appropriate company standards.

Reviewing updating of CAD Manual.

Tel: 01454 203 460

cv@energyrs.co.uk

CAD Manager, Engineering and Mechanical, Aldermaston

mrcas — Fri, 04 Jul 2008 19:50:10 +0000

CAD Manager, Engineering and Mechanical

Reference: 1215082349

ERS have a Client who is looking for a CAD Manager for development of application media for the knowledge management of company configuration records.

Location

Aldermaston

Salary

Competitive salary based on qualifications and experience

Job Responsibilities

A working understanding of relevant British and European Standards and legislative requirements.

Compliance of drawings with company standards.

Maintenance of site drawing standards database.

Provision of internal drawing capabilities to all facilities and projects.

Development of an integrated configuration team, looking at application and storage of configuration data.

Completion of drawings to meet programme requirements.

Carry out surveys to support the completion of drawings.

Preparation of work to pass onto draughtsmen for production of drawings.

Verification of the drawings to appropriate company standards.

Reviewing updating of CAD Manual.

Tel: 01454 203 460

cv@energyrs.co.uk

An Epigrammatic Account of SQL

khatiar2012 — Fri, 04 Jul 2008 17:52:43 +0000

The time gone by of SQL begins in an IBM laboratory in San Jose, California, where on earth SQL was urbanized in the late 1970s. The fundamental pose for Structured Query Language and the language itself is time and again referred to as "sequel." It was in the inauguration built-up for IBM's DB2 item for consumption as a basic criterion of a relational database management system, or RDBMS.. In fact, SQL creates an RDBMS achievable. SQL is a nonprocedural language, in disparity to the procedural or third-generation languages such as COBOL and C that had been created up to that time. The quality that categorizes a DBMS from an RDBMS is that the RDBMS provides a set-oriented database language. For most RDBMS, this set-oriented database language is SQL. Two standards association, the American National Standards Institute and the International Standards Organization, currently prop up SQL standards to exchange. The ANSI-92 standard is the customary for the SQL used throughout this article. Although these standard-making bodies systematize standards for database system designers to tag along, all database products differ from the ANSI standard to some degree. In addition, most systems provide some proprietary extensions to SQL that extend the language into a true procedural language. We have used various RDBMS to prepare the examples in this article to give you an idea of what to expect from the common database systems.

It was an inquiring feeling whether there is a modest background on the evolution of databases and database conjecture would facilitate us value the workings of SQL. Database systems stock up in sequence in every feasible business environment. From outsized pathway databases such as airline proviso systems to a child's baseball card collection, database systems store and hand out the data that we depend on. Until the last few years, large database systems could be run only on large mainframe computers. These machines have traditionally been expensive to design, purchase, and maintain. However, today's generation of powerful, inexpensive workstation computers enables programmers to design software that maintains and distributes data quickly and inexpensively.

Set nocount untuk optimasi Stored Procedures SQL Server

namakuma — Fri, 04 Jul 2008 08:45:00 +0000

Stored procedures dalam SQL server 2000 itu ada 2 macam yaitu system stored procedures dan user stored procedures. User stored procedures terbagi lagi menjadi 3 bagian yaitu :

1. User Defined stored procedures

2. Trigger

3. User defined function

Saya tidak akan membahas mengenai pengertian masing-masing stored procedures diatas, namun saya mencoba memberikan salah satu cara untuk optimasi performance dalam eksekusi stored procedures yaitu dengan menggunakan “Set nocount on”.

Kalo kita jalankan query dengan menggunakan query analyzer sebagai contoh :

SELECT * FROM authors

Message:

(1 row(s) affected)

maka akan selalu muncul pesan statistik berapa banyak jumlah row dari hasil query, didalam stored procedures terutama yang kompleks akan banyak pesan statistik yang di munculkan. Hal ini tentu saja akan menambah waktu proses dan bandwith di jaringan. Oleh sebab itu maka menonaktifkan pesan statistik ini dapat mengurangi proses yg tidak dibutuhkan sehingga performance dari stored procedures akan lebih baik.

create procedure test_coy

begin

declare @i int

create table test_coy_table (

testID int, [user] varchar (50), description varchar(100) )

set @i = 0

while @i < 1000

begin

set @i= @i + 1

insert into test_coy_table (testid,[user]) values(@i, 'user ' + cast(@i as varchar(3)))

end

select * from test_coy_table

end

exec test_coy

message:

(1 row(s) affected)

(1000 row(s) affected)

Contoh pemakaian set nocount :

alter procedure test_coy

begin

set nocount on

declare @i int

create table test_coy_table (

testID int, [user] varchar (50), description varchar(100) )

set @i = 0

while @i < 1000

begin

set @i= @i + 1

insert into test_coy_table (testid,[user]) values(@i, 'user ' + cast(@i as varchar(3)))

end

select * from test_coy_table

end

exec test_coy

message:

Encylopedia of Life

Stephen — Fri, 04 Jul 2008 04:55:34 +0000

A huge and ambitious project to create an online species database. Here is their 'about us' blurb:

The Encyclopedia of Life (EOL) is an ambitious, even audacious project to organize and make available via the Internet virtually all information about life present on Earth. At its heart lies a series of Web sites—one for each of the approximately 1.8 million known species—that provide the entry points to this vast array of knowledge. The entry-point for each site is a species page suitable for the general public, but with several linked pages aimed at more specialized users. The sites sparkle with text and images that are enticing to everyone, as well as providing deep links to specific data.

Try the example page for Cafetaria roenbergensis here.

How to perform an aggregate query using SubSonic 2.1's SqlQuery

Jim — Fri, 04 Jul 2008 01:59:49 +0000

Just recently, I upgraded from SubSonic 2.0 to SubSonic 2.1 RC1. One of the new features is the addition of the SqlQuery class and it's ability to perform aggregate queries (along with many other features).

If you're not familiar with SubSonic, then it is a tool that builds your DAL (Data Abstraction Layer). Each of your database, tables, and rows become classes that you can insert, update, delete, and select from.

By using an ORM instead of embedding SQL statements in your applications, it makes your code easier to test and catch mistakes (such as typos in field or table names). There's pros and cons to ORM versus SQL statements, but this entry won't touch on those issues.

This is an example of how to perform a query to get a count of unique records while grouping by a column.

Sample Data

The below table ("Inventory") will be used throughout these examples.

Year	Make	Model
2000	Acura	Integra
2006	Acura	RSX
2008	BMW	Z4M
2008	Chevrolet	Corvette
2008	Ford	Mustang

How to do this using SQL

Example SQL Code

SELECT Make, Count(0) MakeCount FROM Inventory GROUP BY Make;

SQL Result

Make	MakeCount
Acura	2
BMW	1
Chevrolet	1
Ford	1

How to do this using SubSonic 2.1

Example SubSonic Code

SubSonic.SqlQuery qry = new SubSonic.Select( SubSonic.Aggregate.GroupBy( Inventory.Columns.Make, "Make"), SubSonic.Aggregate.Count( Inventory.Columns.Make, "MakeCount")). From(Inventory.Schema);

myGridView.DataSource = qry.ExecuteReader();

SubSonic Result

Make	MakeCount
Acura	2
BMW	1
Chevrolet	1
Ford	1

How do I do this using a where clause?

Example SQL Code

SELECT Year, Make, Count(0) MakeCount FROM Inventory WHERE Year = 2008 GROUP BY Year, Make;

Example SubSonic Code

SubSonic.SqlQuery qry = new SubSonic.Select( SubSonic.Aggregate.GroupBy( Inventory.Columns.Year, "Year"), SubSonic.Aggregate.GroupBy( Inventory.Columns.Make, "Make"), SubSonic.Aggregate.Count( Inventory.Columns.Make, "MakeCount")). From(Inventory.Schema). Where(Inventory.Columns.Year). IsEqualTo(2008);

myGridView.DataSource = qry.ExecuteReader();

Query Results (both SQL and SubSonic)

Year	Make	MakeCount
2008	BMW	1
2008	Chevrolet	1
2008	Ford	1

My Related Posts: Technology, Database, Programming, .NET

YouTube to disclose user information

VLOGZ — Fri, 04 Jul 2008 01:03:52 +0000

Dismissing privacy concerns, a federal judge overseeing a $1 billion copyright-infringement lawsuit against YouTube has ordered the popular online video-sharing service to disclose who watches which video clips and when/

The database includes information on when each video gets played, which can be used to determine how often a clip is viewed. Attached to each entry is each viewer's unique login ID and the Internet Protocol, or IP, address for that viewer's computer

Lawyers for Google Inc., which owns YouTube, said producing 12 terabytes of data _ equivalent to the text of roughly 12 million books _ would be expensive, time-consuming and a threat to users' privacy.

Will Jon Stewart and Stephen Colbert save us ? The court has yet to rule on Google's requests to question comedians.

YouTube ordered to release user details to Viacom

whitewraithe — Thu, 03 Jul 2008 23:57:00 +0000

This is nothing more than a further invasion of ordinary peoples' privacy. The global Jewish elites (our owners) must be getting really nervous to stoop to this type of surveillance on the web. What are they going to do - arrest anyone who watches an Alex Jones video?

YouTube ordered to hand over user details

By Andrew Ramadge, Technology Reporter

YOUTUBE has been ordered to give up records of each clip watched on the popular video-sharing website, along with the date, time and IP address of each person who watched it, to media giant Viacom.

In a ruling that could have major implications for online privacy around the world, US District Court judge Louis Stanton granted Viacom access to the records as part of its ongoing copyright infringement lawsuit against Google and its subsidiary YouTube.

Each time a video is played, YouTube's "Logging" database records the user ID and IP address of the viewer, the date and time of the request and the ID of the clip – and includes details of videos embedded on websites other than YouTube.

"While the Logging database is large, all of its contents can be copied onto a few 'over-the-shelf' four-terabyte hard drives," Judge Stanton said, in response to Google's claim that providing the data would be too difficult.

"The motion to compel production of all data from the Logging database concerning each time a YouTube video has been viewed on the YouTube website or through embedding on a third-party website is granted."

Viacom sought access to the database in a bid to prove that clips allegedly infringing copyright, such as scenes from TV shows and movies, were more popular than user-generated videos.

Online rights group Electronic Frontier Foundation said the decision stood at odds with US privacy laws and was a "setback to privacy rights".

"The court’s order grants Viacom's request and erroneously ignores the protections of the federal Video Privacy Protection Act (VPPA), and threatens to expose deeply private information about what videos are watched by YouTube users," said a statement on the group's website.

"We urge Viacom to back off this overbroad request and Google to take all steps necessary to challenge this order and protect the rights of its users."

Judge Stanton denied Viacom's requests for access to other Google and YouTube properties, such as the search engine's source code – including the algorithms it uses to provide search results.

Viacom began legal action against YouTube in February 2007, when it issued over 100,000 takedown notices to the website regarding material in breach of copyright.

In March 2007, the media giant instigated a $US1 billion lawsuit against Google and YouTube, alleging that the video-sharing website hosted over 150,000 unauthorised clips that had been viewed more than 1.5 billion times.

Viacom's media empire includes Paramount Pictures, MTV, DreamWorks and Nickelodeon.

It is not known if Google will challenge the decision.

Links

Judge Stanton's ruling on Wired (PDF) – http://blog.wired.com/27bstroke6/files/viacom_yout…
Electronic Frontier Foundation statement – http://www.eff.org/deeplinks/2008/07/court-ruling-…

Source

Gapminder updates

kdewi — Thu, 03 Jul 2008 19:57:03 +0000

Er zijn meer dan 100 nieuwe datasets toegevoegd aan Gapminder, zo valt te lezen op hun blog. Enkele onderwijs-ideeën om Gapminder te introduceren aan je studenten vind je hier. Een handleiding voor Gapminder (doc-formaat, dus te vertalen en aan te passen naar het Nederlands) die je kunt uitprinten voor je leerlingen kun je dan weer hier downloaden.

Start

loonsbury — Thu, 03 Jul 2008 18:41:00 +0000

So this is my first post. Impressed?

I spent most of this morning overcoming the bad decisions of a previous executive director for a client. Note to self: never register a business domain to my own name on my personal Yahoo! account. There are many ways to register a domain name for your company. I think that is one of the most aggravating options.

Getting the business site up soon, we'll see how it turns out.

At the "real" job: mainframe database format changes=bad, mmkay? I'm rewriting every program because somebody couldn't figure out how to get arrayed fields into SQL Server. Someone is slow, I get to pick up the pieces. I know every step of this dance.

Normalisasi dalam Database

siregarbox — Thu, 03 Jul 2008 18:12:19 +0000

Apa sih normalisasi itu?? Apa fungsinya?? Gimana caranya?? Eitss satu-satu donk! Sekarang saya akan mencoba untuk menjelaskan tentang 'Normalisasi dalam Database'. Normalisasi itu teknik untuk mengelompokkan atribut dari suatu relasi sehingga membentuk struktur relasi yang baik(tanpa adanya redudansi). Normalisasi dalam suatu database biasanya hanya mencapai N3(Normalisasi Ketiga), berikut adalah urutan Normalisasi :

1. N1(Normalisasi Pertama)
Mempunyai aturan
~Mendefinisikan primary key
~Tidak ada grup yang berulang
~Semua non-primary key bergantung pada primary key
2. N2(Normalisasi Kedua)
Mempunyai aturan
~Memenuhi aturan N1
~Tidak ada ketergantungan parsial
3. N3(Normalisasi Ketiga)
Mempunyai aturan
~Memenuhi aturan N2
~Tidak ada ketergantungan transitif

Sebagai tambahan normal dalam suatu database seharusnya mencapai bentuk normal tertinggi dan bergerak dari bentuk normal satu dan seterusnya untuk setiap kali membatasi hanya satu jenis redudansi. Jumlah normalisasi seluruhnya ada 5(Lima) dimana 3 bentuk normal pertama menekankan redudansi yang muncul dari Function Dependencies sedangkan N4 dan N5 menekankan redudansi yang muncul dari kasus Multi Valued Dependencies.

Berikut adalah cara singkat melakukan normalisasi
1. Normalisasi Pertama a.k.a N1
Hilangkan duplikasi dengan mencari ketergantungan parsial
2. Normalisasi Kedua a.k.a N2
Field-field yang tergantung pada satu field harus dipisah dengan tepat
3. Normalisasi Ketiga a.k.a N3
Cari hubungan transitif(transitive relation) dimana field non key tergantung pada field non key lainnya

Tabel yang sudah mencapai N3 sudah siap untuk diimplementasikan dalam sebuah proyek, sebenarnya masih ada bentuk normalisasi yang lain yaitu Normalisasi Boyce-Codd dan N4. Jadi sebelum membuat suatu project normalisasikanlah database-mu.

Happy Learning!!

SQL Server Indexes

apcig — Thu, 03 Jul 2008 16:09:51 +0000

Clustered index

- reorders the way records in the table are physically stored

- a table can have only one clustered index

- the leaf nodes of a contain the data pages.

Nonclustered index

- the logical order of the index does not match the physical stored order of the rows on disk

- the leaf nodes of a nonclustered index do not consist of the data pages. Instead, they contain index rows.

Sources: FAQ: clustered vs nonclustered indexes

Database - Affiliazione Database profilati

Directory Affiliazioni — Thu, 03 Jul 2008 15:21:17 +0000

Database - Affiliazioniweb (iscrizione gratuita)

Grazie al Programma di Affiliazioni di Database puoi guadagnare ben 0,05 euro ad ogni Click!

Vendita database aziendali profilati per categoria merceologica e area geografica. Campagne e-mail e fax marketing.

Commissioni: 0,05 euro per ogni Click

Network di appartenza: Affilizioniweb - Clicca qui per iscriverti gratuitamente

Affiliazioni ideale per Newsletter con molti iscritti e per esperti di Fax Marketing.

Monetizza il tuo Blog grazie ai Click di Database (Affiliazioniweb)

Alcuni banner disponibili:

PROTOTYPE MIGRASI DATABASE MICROSOFT SQL SERVER DAN MYSQL KE ORACLE DENGAN KOMPONEN ADO, MYSQLDAC DAN ODAC

prasetyo2008 — Thu, 03 Jul 2008 11:28:13 +0000

Database merupakan jantung dalam sebuah sistem informasi yang mengolah data dalam sebuah instansi. Beberapa hal yang menjadi pertimbangan utama dalam pemilihan sebuah perangkat lunak database adalah kemudahan dalam penggunaan dan perawatan, kelengkapan fasilitas yang disediakan, keamanan akses data, lisensi pembelian perangkat lunak, kesesuaian dengan sistem operasi yang digunakan. Dengan semakin berkembangnya kebutuhan informasi instansi tersebut maka perangkat lunak database yang digunakan juga harus dapat memenuhi kebutuhan yang diharapkan oleh instansi tersebut.

Jika perangkat lunak yang digunakan oleh instansi tersebut tidak dapat memenuhi kebutuhan yang diharapkan maka lebih baik jika database tersebut dipindahkan dari mesin database lama ke mesin database baru yang dapat memenuhi kebutuhan yang diharapkan. Untuk memindahkan database tersebut diperlukan sebuah sistem yang dapat digunakan untuk memindahkan database dari mesin database yang lama ke mesin database yang baru.

Dengan adanya sistem pemindah database maka pemindahan database menjadi lebih mudah, karena proses pemindahan database yang meliputi pemindahan struktur dan isi tabel dilakukan secara otomatis oleh sistem tersebut. Hasilnya berupa database yang sama di mesin database yang baru.

Kata kunci : database, pemindahan, Microsoft SQL Server, MySQL, Oracle, sistem, informasi.

the "2nd"

Niko — Thu, 03 Jul 2008 10:18:05 +0000

seken...

apa yg saia kerjakan hari inih???

pagi-hingga rada siang : browsing... mencari "literatur tertentu" untuk dijadikan sumber, hehe.. kemaren kan dimensi Time ama Geography sudah, sekarang mulai ngeniatin buat ngedesain control flow etl dimensi product, udah sih kemaren sore tp daripada pagi ini saia dikira tidak bekerja, yah dimasukin disini sajah..

ada sedikit masalah di database sumbernya... huh, yang penting bukan salah desain saia.. hwahaha..

sebelum siang : ngetes desain etl tersebut.... banyak yang saia akalin biar bisa dipake (seenggaknya sampai masa shareware sistem ini habis ;p) Hore! hore! dimensi sudah... loncat ke fact... ternyata eh ternyta, sudah waktunya lunch..

lunch : ya makan lah.... (anjir, dahar kalaka nyieun nyeri beuteung, haram jadah, engke deui mah moal seueur2 nyandak sambelna...)

abis lunch-sore : fact... fact... fact... dimulai dari yang paling gampang.. fact debit... huhu mantap datanya cuma dikit (cuma 2 bp yang ngasih data debit!) mikir > desain > tes > beres (alhamdulilah!)

sore bgt : mikirin desain fact selanjutnya, yaitu fact sales... huhu bakalan sengsara sampai ke LDBD kayanya nih.... desainnya belom selese, kepotong ama (B/KP)logging...

ah sudah... hayang buru2 m*d*l yeuh... rek nonton mamah jeung aa oge *kemarin saia ketinggalan nonton mamah dan aa, hiks2..*

Saia juga dapat pelajaran hari ini bahwa menurut sunnah nabi ternyata laki2 (seperti saya!) ketika kencing tidak boleh memegang "albert" dengan tangan kanan (bukan jg pakai kaki kanan) tapi harus pakai tangan kiri... heu kan lumayan kencing aja dapet pahala...

SQL Injection Attacks by Example

coolvinay — Thu, 03 Jul 2008 09:35:04 +0000

"SQL Injection" is subset of the an unverified/unsanitized user input vulnerability ("buffer overflows" are a different subset), and the idea is to convince the application to run SQL code that was not intended. If the application is creating SQL strings naively on the fly and then running them, it's straightforward to create some real surprises.

We'll note that this was a somewhat winding road with more than one wrong turn, and others with more experience will certainly have different -- and better -- approaches. But the fact that we were successful does suggest that we were not entirely misguided.

There have been other papers on SQL injection, including some that are much more detailed, but this one shows the rationale of discovery as much as the process of exploitation

The Target Intranet

This appeared to be an entirely custom application, and we had no prior knowledge of the application nor access to the source code: this was a "blind" attack. A bit of poking showed that this server ran Microsoft's IIS 6 along with ASP.NET, and this suggested that the database was Microsoft's SQL server: we believe that these techniques can apply to nearly any web application backed by any SQL server.

The login page had a traditional username-and-password form, but also an email-me-my-password link; the latter proved to be the downfall of the whole system.

When entering an email address, the system presumably looked in the user database for that email address, and mailed something to that address. Since my email address is not found, it wasn't going to send me anything.

So the first test in any SQL-ish form is to enter a single quote as part of the data: the intention is to see if they construct an SQL string literally without sanitizing. When submitting the form with a quote in the email address, we get a 500 error (server failure), and this suggests that the "broken" input is actually being parsed literally. Bingo.

We speculate that the underlying SQL code looks something like this:

SELECT fieldlist
  FROM table
 WHERE field = '$EMAIL';

Here, $EMAIL is the address submitted on the form by the user, and the larger query provides the quotation marks that set it off as a literal string. We don't know the specific names of the fields or table involved, but we do know their nature, and we'll make some good guesses later.

When we enter steve@unixwiz.net' - note the closing quote mark - this yields constructed SQL:

SELECT fieldlist
  FROM table
 WHERE field = 'steve@unixwiz.net'';

when this is executed, the SQL parser find the extra quote mark and aborts with a syntax error. How this manifests itself to the user depends on the application's internal error-recovery procedures, but it's usually different from "email address is unknown". This error response is a dead giveaway that user input is not being sanitized properly and that the application is ripe for exploitation.

Since the data we're filling in appears to be in the WHERE clause, let's change the nature of that clause in an SQL legal way and see what happens. By entering anything' OR 'x'='x, the resulting SQL is:

SELECT fieldlist
  FROM table
 WHERE field = 'anything' OR 'x'='x';

Because the application is not really thinking about the query - merely constructing a string - our use of quotes has turned a single-component WHERE clause into a two-component one, and the 'x'='x' clause is guaranteed to be true no matter what the first clause is (there is a better approach for this "always true" part that we'll touch on later).

But unlike the "real" query, which should return only a single item each time, this version will essentially return every item in the members database. The only way to find out what the application will do in this circumstance is to try it. Doing so, we were greeted with:

Your login information has been mailed to random.person@example.com.

Our best guess is that it's the first record returned by the query, effectively an entry taken at random. This person really did get this forgotten-password link via email, which will probably come as surprise to him and may raise warning flags somewhere.

We now know that we're able to manipulate the query to our own ends, though we still don't know much about the parts of it we cannot see. But we have observed three different responses to our various inputs:

"Your login information has been mailed to email"
"We don't recognize your email address"
Server error

The first two are responses to well-formed SQL, while the latter is for bad SQL: this distinction will be very useful when trying to guess the structure of the query.

Schema field mapping

The first steps are to guess some field names: we're reasonably sure that the query includes "email address" and "password", and there may be things like "US Mail address" or "userid" or "phone number". We'd dearly love to perform a SHOW TABLE, but in addition to not knowing the name of the table, there is no obvious vehicle to get the output of this command routed to us.

So we'll do it in steps. In each case, we'll show the whole query as we know it, with our own snippets shown specially. We know that the tail end of the query is a comparison with the email address, so let's guess email as the name of the field:

SELECT fieldlist
  FROM table
 WHERE field = 'x' AND email IS NULL; --';

The intent is to use a proposed field name (email) in the constructed query and find out if the SQL is valid or not. We don't care about matching the email address (which is why we use a dummy 'x'), and the -- marks the start of an SQL comment. This is an effective way to "consume" the final quote provided by application and not worry about matching them.

If we get a server error, it means our SQL is malformed and a syntax error was thrown: it's most likely due to a bad field name. If we get any kind of valid response, we guessed the name correctly. This is the case whether we get the "email unknown" or "password was sent" response.

Note, however, that we use the AND conjunction instead of OR: this is intentional. In the SQL schema mapping phase, we're not really concerned with guessing any particular email addresses, and we do not want random users inundated with "here is your password" emails from the application - this will surely raise suspicions to no good purpose. By using the AND conjunction with an email address that couldn't ever be valid, we're sure that the query will always return zero rows and never generate a password-reminder email.

Submitting the above snippet indeed gave us the "email address unknown" response, so now we know that the email address is stored in a field email. If this hadn't worked, we'd have tried email_address or mail or the like. This process will involve quite a lot of guessing.

Next we'll guess some other obvious names: password, user ID, name, and the like. These are all done one at a time, and anything other than "server failure" means we guessed the name correctly.

SELECT fieldlist
  FROM table
 WHERE email = 'x' AND userid IS NULL; --';

As a result of this process, we found several valid field names:

email
passwd
login_id
full_name

There are certainly more (and a good source of clues is the names of the fields on forms), but a bit of digging did not discover any. But we still don't know the name of the table that these fields are found in - how to find out?

Finding the table name

The application's built-in query already has the table name built into it, but we don't know what that name is: there are several approaches for finding that (and other) table names. The one we took was to rely on a subselect.

A standalone query of

SELECT COUNT(*) FROM tabname

Returns the number of records in that table, and of course fails if the table name is unknown. We can build this into our string to probe for the table name:

SELECT email, passwd, login_id, full_name
  FROM table
 WHERE email = 'x' AND 1=(SELECT COUNT(*) FROM tabname); --';

We don't care how many records are there, of course, only whether the table name is valid or not. By iterating over several guesses, we eventually determined that members was a valid table in the database. But is it the table used in this query? For that we need yet another test using table.field notation: it only works for tables that are actually part of this query, not merely that the table exists.

SELECT email, passwd, login_id, full_name
  FROM members
 WHERE email = 'x' AND members.email IS NULL; --';

When this returned "Email unknown", it confirmed that our SQL was well formed and that we had properly guessed the table name. This will be important later, but we instead took a different approach in the interim.

Finding some users

At this point we have a partial idea of the structure of the members table, but we only know of one username: the random member who got our initial "Here is your password" email. Recall that we never received the message itself, only the address it was sent to. We'd like to get some more names to work with, preferably those likely to have access to more data.

The first place to start, of course, is the company's website to find who is who: the "About us" or "Contact" pages often list who's running the place. Many of these contain email addresses, but even those that don't list them can give us some clues which allow us to find them with our tool.

The idea is to submit a query that uses the LIKE clause, allowing us to do partial matches of names or email addresses in the database, each time triggering the "We sent your password" message and email. Warning: though this reveals an email address each time we run it, it also actually sends that email, which may raise suspicions. This suggests that we take it easy.

We can do the query on email name or full name (or presumably other information), each time putting in the % wildcards that LIKE supports:

SELECT email, passwd, login_id, full_name
  FROM members
 WHERE email = 'x' OR full_name LIKE '%Bob%';

Keep in mind that even though there may be more than one "Bob", we only get to see one of them: this suggests refining our LIKE clause narrowly.

Ultimately, we may only need one valid email address to leverage our way in.

Brute-force password guessing

One can certainly attempt brute-force guessing of passwords at the main login page, but many systems make an effort to detect or even prevent this. There could be logfiles, account lockouts, or other devices that would substantially impede our efforts, but because of the non-sanitized inputs, we have another avenue that is much less likely to be so protected.

We'll instead do actual password testing in our snippet by including the email name and password directly. In our example, we'll use our victim, bob@example.com and try multiple passwords.

SELECT email, passwd, login_id, full_name
  FROM members
 WHERE email = 'bob@example.com' AND passwd = 'hello123';

This is clearly well-formed SQL, so we don't expect to see any server errors, and we'll know we found the password when we receive the "your password has been mailed to you" message. Our mark has now been tipped off, but we do have his password.

This procedure can be automated with scripting in perl, and though we were in the process of creating this script, we ended up going down another road before actually trying it.

The database isn't readonly

So far, we have done nothing but query the database, and even though a SELECT is readonly, that doesn't mean that SQL is. SQL uses the semicolon for statement termination, and if the input is not sanitized properly, there may be nothing that prevents us from stringing our own unrelated command at the end of the query.

The most drastic example is:

SELECT email, passwd, login_id, full_name
  FROM members
 WHERE email = 'x'; DROP TABLE members; --';  -- Boom!

The first part provides a dummy email address -- 'x' -- and we don't care what this query returns: we're just getting it out of the way so we can introduce an unrelated SQL command. This one attempts to drop (delete) the entire members table, which really doesn't seem too sporting.

This shows that not only can we run separate SQL commands, but we can also modify the database. This is promising.

Adding a new member

Given that we know the partial structure of the members table, it seems like a plausible approach to attempt adding a new record to that table: if this works, we'll simply be able to login directly with our newly-inserted credentials.

This, not surprisingly, takes a bit more SQL, and we've wrapped it over several lines for ease of presentation, but our part is still one contiguous string:

SELECT email, passwd, login_id, full_name
  FROM members
 WHERE email = 'x';
        INSERT INTO members ('email','passwd','login_id','full_name') 
        VALUES ('steve@unixwiz.net','hello','steve','Steve Friedl');--';

Even if we have actually gotten our field and table names right, several things could get in our way of a successful attack:

We might not have enough room in the web form to enter this much text directly (though this can be worked around via scripting, it's much less convenient).
The web application user might not have INSERT permission on the members table.
There are undoubtedly other fields in the members table, and some may require initial values, causing the INSERT to fail.
Even if we manage to insert a new record, the application itself might not behave well due to the auto-inserted NULL fields that we didn't provide values for.
A valid "member" might require not only a record in the members table, but associated information in other tables (say, "accessrights"), so adding to one table alone might not be sufficient.

In the case at hand, we hit a roadblock on either #4 or #5 - we can't really be sure -- because when going to the main login page and entering in the above username + password, a server error was returned. This suggests that fields we did not populate were vital, but nevertheless not handled properly.

A possible approach here is attempting to guess the other fields, but this promises to be a long and laborious process: though we may be able to guess other "obvious" fields, it's very hard to imagine the bigger-picture organization of this application.

We ended up going down a different road.

Mail me a password

We then realized that though we are not able to add a new record to the members database, we can modify an existing one, and this proved to be the approach that gained us entry.

From a previous step, we knew that bob@example.com had an account on the system, and we used our SQL injection to update his database record with our email address:

SELECT email, passwd, login_id, full_name
  FROM members
 WHERE email = 'x';
      UPDATE members
      SET email = 'steve@unixwiz.net'
      WHERE email = 'bob@example.com';

After running this, we of course received the "we didn't know your email address", but this was expected due to the dummy email address provided. The UPDATE wouldn't have registered with the application, so it executed quietly.

We then used the regular "I lost my password" link - with the updated email address - and a minute later received this email:

From: system@example.com
To: steve@unixwiz.net
Subject: Intranet login

This email is in response to your request for your Intranet log in information.
Your User ID is: bob
Your password is: hello

Now it was now just a matter of following the standard login process to access the system as a high-ranked MIS staffer, and this was far superior to a perhaps-limited user that we might have created with our INSERT approach.

We found the intranet site to be quite comprehensive, and it included - among other things - a list of all the users. It's a fair bet that many Intranet sites also have accounts on the corporate Windows network, and perhaps some of them have used the same password in both places. Since it's clear that we have an easy way to retrieve any Intranet password, and since we had located an open PPTP VPN port on the corporate firewall, it should be straightforward to attempt this kind of access.

We had done a spot check on a few accounts without success, and we can't really know whether it's "bad password" or "the Intranet account name differs from the Windows account name". But we think that automated tools could make some of this easier.

Other Approaches

In this particular engagement, we obtained enough access that we did not feel the need to do much more, but other steps could have been taken. We'll touch on the ones that we can think of now, though we are quite certain that this is not comprehensive.

We are also aware that not all approaches work with all databases, and we can touch on some of them here.

Use xp_cmdshell: Microsoft's SQL Server supports a stored procedure xp_cmdshell that permits what amounts to arbitrary command execution, and if this is permitted to the web user, complete compromise of the webserver is inevitable.; What we had done so far was limited to the web application and the underlying database, but if we can run commands, the webserver itself cannot help but be compromised. Access to xp_cmdshell is usually limited to administrative accounts, but it's possible to grant it to lesser users.
Map out more database structure: Though this particular application provided such a rich post-login environment that it didn't really seem necessary to dig further, in other more limited environments this may not have been sufficient.; Being able to systematically map out the available schema, including tables and their field structure, can't help but provide more avenues for compromise of the application.; One could probably gather more hints about the structure from other aspects of the website (e.g., is there a "leave a comment" page? Are there "support forums"?). Clearly, this is highly dependent on the application and it relies very much on making good guesses.

Mitigations

We believe that web application developers often simply do not think about "surprise inputs", but security people do (including the bad guys), so there are three broad approaches that can be applied here.

Sanitize the input

It's absolutely vital to sanitize user inputs to insure that they do not contain dangerous codes, whether to the SQL server or to HTML itself. One's first idea is to strip out "bad stuff", such as quotes or semicolons or escapes, but this is a misguided attempt. Though it's easy to point out some dangerous characters, it's harder to point to all of them.

The language of the web is full of special characters and strange markup (including alternate ways of representing the same characters), and efforts to authoritatively identify all "bad stuff" are unlikely to be successful.

Instead, rather than "remove known bad data", it's better to "remove everything but known good data": this distinction is crucial. Since - in our example - an email address can contain only these characters:

abcdefghijklmnopqrstuvwxyz
ABCDEFGHIJKLMNOPQRSTUVWXYZ
0123456789
@.-_+

There is really no benefit in allowing characters that could not be valid, and rejecting them early - presumably with an error message - not only helps forestall SQL Injection, but also catches mere typos early rather than stores them into the database.

Sidebar on email addresses

It's important to note here that email addresses in particular are troublesome to validate programmatically, because everybody seems to have his own idea about what makes one "valid", and it's a shame to exclude a good email address because it contains a character you didn't think about.

The only real authority is RFC 2822 (which encompasses the more familiar RFC822), and it includes a fairly expansive definition of what's allowed. The truly pedantic may well wish to accept email addresses with ampersands and asterisks (among other things) as valid, but others - including this author - are satisfied with a reasonable subset that includes "most" email addresses.

Those taking a more restrictive approach ought to be fully aware of the consequences of excluding these addresses, especially considering that better techniques (prepare/execute, stored procedures) obviate the security concerns which those "odd" characters present.

Be aware that "sanitizing the input" doesn't mean merely "remove the quotes", because even "regular" characters can be troublesome. In an example where an integer ID value is being compared against the user input (say, a numeric PIN):

SELECT fieldlist
  FROM table
 WHERE id = 23 OR 1=1;  -- Boom! Always matches!

In practice, however, this approach is highly limited because there are so few fields for which it's possible to outright exclude many of the dangerous characters. For "dates" or "email addresses" or "integers" it may have merit, but for any kind of real application, one simply cannot avoid the other mitigations.

Escape/Quotesafe the input

Even if one might be able to sanitize a phone number or email address, one cannot take this approach with a "name" field lest one wishes to exclude the likes of Bill O'Reilly from one's application: a quote is simply a valid character for this field.

One includes an actual single quote in an SQL string by putting two of them together, so this suggests the obvious - but wrong! - technique of preprocessing every string to replicate the single quotes:

SELECT fieldlist
  FROM customers
 WHERE name = 'Bill O''Reilly';  -- works OK

However, this naïve approach can be beaten because most databases support other string escape mechanisms. MySQL, for instance, also permits \' to escape a quote, so after input of \'; DROP TABLE users; -- is "protected" by doubling the quotes, we get:

SELECT fieldlist
  FROM customers
 WHERE name = '\''; DROP TABLE users; --';  -- Boom!

The expression '\'' is a complete string (containing just one single quote), and the usual SQL shenanigans follow. It doesn't stop with backslashes either: there is Unicode, other encodings, and parsing oddities all hiding in the weeds to trip up the application designer.

Getting quotes right is notoriously difficult, which is why many database interface languages provide a function that does it for you. When the same internal code is used for "string quoting" and "string parsing", it's much more likely that the process will be done properly and safely.

Some examples are the MySQL function mysql_real_escape_string() and perl DBD method $dbh->quote($value).

These methods must be used.

Use bound parameters (the PREPARE statement)

Though quotesafing is a good mechanism, we're still in the area of "considering user input as SQL", and a much better approach exists: bound parameters, which are supported by essentially all database programming interfaces. In this technique, an SQL statement string is created with placeholders - a question mark for each parameter - and it's compiled ("prepared", in SQL parlance) into an internal form.

Later, this prepared query is "executed" with a list of parameters:

Example in perl

$sth = $dbh->prepare("SELECT email, userid FROM members WHERE email = ?;");

$sth->execute($email);

Thanks to Stefan Wagner, this demonstrates bound parameters in Java:

Insecure version

Statement s = connection.createStatement();
ResultSet rs = s.executeQuery("SELECT email FROM member WHERE name = "
                             + formField); // *boom*

Secure version

PreparedStatement ps = connection.prepareStatement(
    "SELECT email FROM member WHERE name = ?");
ps.setString(1, formField);
ResultSet rs = ps.executeQuery();

Here, $email is the data obtained from the user's form, and it is passed as positional parameter #1 (the first question mark), and at no point do the contents of this variable have anything to do with SQL statement parsing. Quotes, semicolons, backslashes, SQL comment notation - none of this has any impact, because it's "just data". There simply is nothing to subvert, so the application is be largely immune to SQL injection attacks.

There also may be some performance benefits if this prepared query is reused multiple times (it only has to be parsed once), but this is minor compared to the enormous security benefits. This is probably the single most important step one can take to secure a web application.

Limit database permissions and segregate users

In the case at hand, we observed just two interactions that are made not in the context of a logged-in user: "log in" and "send me password". The web application ought to use a database connection with the most limited rights possible: query-only access to the members table, and no access to any other table.

The effect here is that even a "successful" SQL injection attack is going to have much more limited success. Here, we'd not have been able to do the UPDATE request that ultimately granted us access, so we'd have had to resort to other avenues.

Once the web application determined that a set of valid credentials had been passed via the login form, it would then switch that session to a database connection with more rights.

It should go almost without saying that sa rights should never be used for any web-based application.

Use stored procedures for database access

When the database server supports them, use stored procedures for performing access on the application's behalf, which can eliminate SQL entirely (assuming the stored procedures themselves are written properly).

By encapsulating the rules for a certain action - query, update, delete, etc. - into a single procedure, it can be tested and documented on a standalone basis and business rules enforced (for instance, the "add new order" procedure might reject that order if the customer were over his credit limit).

For simple queries this might be only a minor benefit, but as the operations become more complicated (or are used in more than one place), having a single definition for the operation means it's going to be more robust and easier to maintain.

Note: it's always possible to write a stored procedure that itself constructs a query dynamically: this provides no protection against SQL Injection - it's only proper binding with prepare/execute or direct SQL statements with bound variables that provide this protection.

Isolate the webserver

Even having taken all these mitigation steps, it's nevertheless still possible to miss something and leave the server open to compromise. One ought to design the network infrastructure to assume that the bad guy will have full administrator access to the machine, and then attempt to limit how that can be leveraged to compromise other things.

For instance, putting the machine in a DMZ with extremely limited pinholes "inside" the network means that even getting complete control of the webserver doesn't automatically grant full access to everything else. This won't stop everything, of course, but it makes it a lot harder.

Configure error reporting

The default error reporting for some frameworks includes developer debugging information, and this cannot be shown to outside users. Imagine how much easier a time it makes for an attacker if the full query is shown, pointing to the syntax error involved.

This information is useful to developers, but it should be restricted - if possible - to just internal users.

Note that not all databases are configured the same way, and not all even support the same dialect of SQL (the "S" stands for "Structured", not "Standard"). For instance, most versions of MySQL do not support subselects, nor do they usually allow multiple statements: these are substantially complicating factors when attempting to penetrate a network.

source: http://www.unixwiz.net/techtips/sql-injection.html

Operasi Dasar Basis Data (Database)

mrkopetz — Thu, 03 Jul 2008 06:59:43 +0000

Operasi-operasi dasar dapat kita lakukan terhadap basis data meliputi:

Pembuatan basis data baru (create database), yang identik dengan pembuatan lemari arsip yang baru.
Penghapusan basis data (drop database), yang identik dengan pengrusakan lemari arsip (sekaligus beserta isinya jika ada).
Pembuatan file/tabel baru ke suatu basis data (create table), yang identik dengan penambahan map arsip baru ke sebuah lemari arsip yang telah ada.
Penghapusan file/tabel dari suatu basis data (drop table), yang identik dengan perusakan map arsip lama yang ada di sebuah lemari arsip.
Penambahan/pengisian data baru ke sebuah file/tabel di sebuah basis data (insert), yang identik dengan penambahan lembaran arsip ke sebuah map arsip.
Pengambilan data dari sebuah file/tabel (retrieve/search/select), yang identik dengan pencarian lembaran arsip dari sebuah map arsip.
Pengubahan data dari file/tabel (update), yang identik dengan perbaikan isi lembaran arsip yang ada di sebuah lemari arsip.
Penghapusan data dari sebuah file/tabel (delete), yang identik dengan penghapusan sebuahlembaran arsip yang ada di sebuah map arsip.

Sebuah basis data umumnya dibuat untuk mewakili sebuah semesta data yang spesifik. Misalnya, basis data kepegawaian yang mewakili semesta pegawai, basis data akademik, basis data inventori (pergudangan), dan lain sebagainya. Sementara dalam basis data terdapat file-file/tabel-tabel yang berisi data. Misalnya pada basis data kepegawaian kita dapat menempatkan file/tabel biodata_pegawai, jabatan, keluarga, penilaian_kerja, dan seterusnya.

Sumber:

Fathansyah, Ir. Basis Data. Bandung: Penerbit Informatika Bandung, 1999

Strategy for eliminating redundant legacy applications

davidstech — Thu, 03 Jul 2008 05:20:53 +0000

Application retirement is a viable strategy for eliminating redundant legacy applications, thereby improving operational efficiency and reducing costs. By reducing the number of applications within the IT infrastructure, DBAs can concentrate on maintaining critical business applications with the highest value to the organization instead of compiling data from disparate sources. Solix Application Sunsetting and Migration is part of the Solix Enterprise Data Management Suite (Solix EDMS) that enables centralized management of data classification and security policies. At the core of the Solix EDMS is the Solix Metadata Manager, which manages multiple applications and data types across at hardware platforms through a unified policy manager.