Dynamically creating DSN entries

There's a good article about creating dynamic DSN entries, which can be put together with the blog I wrote about copying tables to MS Access. This way you don't need to create the DSN manually, and can also clean up when the copy is done by automatically removing the DSN entry.

The article is here: http://www.codeproject.com/cs/database/DSNAdmin.asp

How to quickly copy tables from an ODBC source to MS Access within a C# project

Introduction

In a recent project I worked on, I ran across a pretty serious problem that seemingly had no solution. The problem was simple: I needed to copy a fairly large amount of table from a table in an SQL Server database to a table in an MS Access database. No matter which approach I followed, however, the speeds at which the copy was being executed were far too low and simply unacceptable.

My frame of reference was MS Access itself, which has the ability to import tables from an ODBC source. When I compared the time it took to perform this import, it consistently beat the other tests by a wide margin of at least twenty to one.

It sounds huge. I know. I do not have the exact times it took for each of the tests, but they were all much longer than doing a simple import through MS Access. The comparison tests I conducted were:

• Using Enterprise Manager
• Using SQL Server Management Studio
• Using SQL Server Information Services
• Using BCP (although this can only export to a text file or SQL Server binary file). The BCP process alone was able to match the time record achieved by the MS Access import, but useless since it could not export to MS Access.

The ultimate solution, then, was to find a way to have MS Access perform the copy all the while running within a C# project. The biggest problem with this was that including support for MS Access in a .NET project automatically means including COM, and a lot of people cringe at the thought. This article, then, discusses the implementation of an SQL server to MS Access copy process that is performed by MS Access, runs within a C# project, and minimizes the impact COM has on the overall application by using late binding.

Embedding a COM object in .NET

It isn’t pretty, but it’s possible. It will make your setup and installation more complex, but will add functionality that may otherwise not be available from within the .NET framework. The bottom line is that it is up to you on whether you want to include COM support in your project, and you’ll have to weigh the pros and cons and then decide.

In my case, there wasn’t much of a choice. I could either leave my project spic and span and let the copy take almost an hour, or find an elegant way to implement COM and shrink the copy process down to a couple of minutes. The choice was pretty clear.

There are basically two ways to use a COM object in your project:

Early binding

In this case, you add a reference to the COM object in question to your project, and benefit from the fact that Visual Studio can (in some cases) offer intellisense on the classes, methods and properties of this object. For screen controls, you also get the ability to add them to your forms in the screen designer. On the down side, however, you can no longer just copy the project files from your bin\debug or bin\release folder to another machine and run the program. You now need a setup project that will copy and register the COM controls on the target machine as well as copy your project files.

Late binding

If you are guaranteed that the COM object in question is already installed on the target computer, and can do away with such niceties as intellisense, you can write your code so that it hooks up to the already-installed COM control and use it in pretty much the same way. The down side, obviously, is that you are not guaranteed your code will always work (because you cannot be 100% certain the COM object is already installed on the target machine), and you will not have access to intellisense or the screen designer.

Importing a table

When importing a table into MS Access, the application lets you do so by going to File\Get external data\Import. You then choose a source such as an ODBC connection, select the database and table, and that’s it. The table gets copied over (albeit without any keys or indexes, which you then have to rebuild).

This functionality is very much available through COM automation, and can therefore be used in a C# project. The problem is that the code is only available from within an Access module (although I’m sure one the readers will find a way to access this function directly from C#). So to get to it, you need to write a VB function that will import a table. The C# project will then need to load the Access database and execute the function which in turns performs the import.

The only drawback to this method is that it is not asynchronous (although you could add threading on the C# side) and has no way to indicate progress.

The VB function you’ll need to put into MS Access is quite simple, and basically calls the TransferDatabase method by passing it a DSN (pointing to the source database), a source table name and target table name. The code is as follows:

Public Function Import(dsnName As String, sourceTableName As String, targetTableName As String)
‘ if the table already existsm, delete it.
   On Error GoTo CopyTable
   DoCmd.DeleteObject acTable, targetTableName
CopyTable:
   DoCmd.TransferDatabase _
   acImport, _
   "ODBC Database", _
   "ODBC;DSN=" + dsnName, _
   acTable, _
   sourceTableName, _
   targetTableName
End Function

And then the C# code:

object accessObject = null;
try
{
   accessObject = Activator.CreateInstance(Type.GetTypeFromProgID("Access.Application"));

   accessObject.GetType().InvokeMember(
      "OpenCurrentDatabase",
      System.Reflection.BindingFlags.Default  System.Reflection.BindingFlags.InvokeMethod,
      null,
      accessObject,
      new Object[] { "AccessDbase.mdb" });

   accessObject.GetType().InvokeMember(
      "Run",
      System.Reflection.BindingFlags.Default  System.Reflection.BindingFlags.InvokeMethod,
      null,
      accessObject,
      new Object[] { "Import", "DSN Name", "Source table name", "Target table name" });

   accessObject.GetType().InvokeMember(
      "CloseCurrentDatabase",
      System.Reflection.BindingFlags.Default  System.Reflection.BindingFlags.InvokeMethod,
      null,
      accessObject,
      null);

   MessageBox.Show("Copy succeeded.");
}
catch (Exception ex)
{
   string message = ex.Message;
   while (ex.InnerException != null)
   {
      ex = ex.InnerException;
      message += "\r\n----\r\n" + ex.Message;
   }
   MessageBox.Show(message);
}
finally
{
   if (accessObject != null)
   {
      System.Runtime.InteropServices.Marshal.ReleaseComObject(accessObject);
      accessObject = null;
   }
}

All we’re doing here is creating an instance of Access.Application, opening a database, executing the VB module to perform the import, closing the database and cleaning up.

Note that for the copy to work, you need to specify several pieces of information:

• The fully qualified name of the Access database
• The name of an ODBC DSN created in your control panel that points to the source database (i.e. where the table will be copied from)
• The name of the table as it appears in the source database
• The name you want to assign to the table once it’s been copied (can be the same as the source).

Finally, it should be noted that the copy process creates the table with no keys or indexes, so these will have to be rebuilt after the copy is complete. This is very easily achieved, however, by executing SQL "ALTER TABLE " commands through ADO. There are plenty of articles out there that describe this so I won’t get into it here.

Credits and further info
This article may not seem very big, but it is the culmination of a massive headache and lot of needle hunting (in a haystack of needles) on the net. I found several articles out there that helped out, and these are some of them:
http://www.thescripts.com/forum/thread255310.html - executing an MS Access module from C#
http://support.microsoft.com/kb/306683 - executing MS Access modules from within C#. Note that the method described here uses early binding, which means you’re adding COM references to your project and you are therefore complicating your setup/installation package.
http://www.codeproject.com/cs/database/mdbcompact_latebind.asp - very good article by Alexander Yumashev describing how to compact and repair an MS Access database from within C#. More importantly, it overcomes the problem in the Microsoft article mentioned above by using late binding.

In conclusion
MS Access is being put out to pasture out for many good reasons, but it’s still very much in use. So if you’re one of those people who still has to copy large amounts of data from another database source to MS Access, need to do in a C# project, and want a clean way to do it, then I think this article will help. If you don’t fall in that criteria, though, then I hope you at least had an interesting read and kept this in the back of your mind for future use.

A sample project is available for testing here: http://www.codeproject.com/useritems/MSAccessTableCopyInCSharp.asp

A very fast table copy from SQL Server to MS Access

New article coming up ...

I've recently fought (and won) a battle to get the fastest possible copy of a table from SQL Server to MS Access within a C# project. I tried the usual things like SSIS and ADO.net, but the fastest thing was the import utility within Access itself. I finally go it to work properly, and it flies (it's an average of 20 - yes 20 - times faster than copying through SSIS or other widely used methods). I'll post the article in the coming days.

A C# central logging mechanism using the Observer and Singleton patterns

Introduction

Logging is something any respectable application needs to do. Whether it's to keep track of what the application is doing under the hood (for possible trouble shooting), to show a list of events that we might need to inform the user of, or simply to keep a history of events, logging is as necessary to the programming world as note taking is to a class room. If you don't do it, you might be able to get by for the moment. Just hope you never need to refer back to something that happened a few minutes, hours or days ago.

This article shows an excellent mechanism for adding logging to a C# project. It is extremely powerful and versatile, yet very simple to implement and use. It uses two very common programming patterns: the singleton and the observer.

Background

The ideas and code shown in here are not rocket science, so as long as you have at least a basic idea of object oriented programming, and have been playing around with C# for a while, I'd say you're safe.

This having been said, it is always a good idea to brush up on a couple of things if you want to get the most out of this. I would suggest reading up on patterns, and focusing on some of the more common ones such as the Singleton and Observer.

To keep it brief and concise, however, I'll delve into these two for a bit and give you an idea of what they're about.

The Singleton pattern

In layman's terms ...

Probably the most widely known and one of simplest patterns, the singleton is a mechanism that ensures all objects in your process have access to a shared and single set of data and methods.
Imagine an intersection where several roads meet, and where the cars need to ensure they do not crash into each other when crossing this intersection. Assuming there were no traffic lights to guide them, we would need a police officer standing in the middle to direct traffic. This officer would be a real life approximation of a singleton.

The officer would receive visual queues from all roads, telling him which one is filling up the most with cars. With all this information, he would be able to decide which road to let the cars come through, and which roads to block.

This would be significantly more complex if there were several officers controlling the intersection, where they would all be receiving visual queues from the roads, would have to communicate with each other and then collectively decide which road to let traffic come from.
Similarly, a singleton object is something shared by all the objects in the system. For the purposes of the logging mechanism, the singleton is the one and only object to which all other objects will send the information they wish to have logged.

Not only does this centralize and simplify control of the logging mechanism, this also gives the developer an excellent way to provide uniform formatting to the log output. You can add timestamps, titles, parameter information and more in just one place. None of the objects that need information to be logged need be concerned with this, as the singleton logger takes care of it on its own.

Putting this in code ...

When regular classes need to be used, they are instantiated as an object and then used. A singleton class, however, does not allow anyone except itself to instantiate it. This guarantees that only one copy of this object will ever run, and all objects in the program will be accessing this one copy.

To accomplish this, the logger has only one private constructor and a private variable of its own type. The only way to really get a hold of this object from the outside then is to instantiate the Logger class and equate the new object to the static handle exposed in the class. The Instance property then checks to see if the private mLogger object was ever created, and if it was not, this is the only place where Logger will ever get instantiated.

Logger
{
   private object mLock;
   private static Logger mLogger = null;
   // the public Instance property everyone uses to access the Logger
   public static Logger Instance
   {
      get
      {
         // If this is the first time we're referring to the
         // singleton object, the private variable will be null.
         if (mLogger == null)
         {
            // for thread safety, lock an object when
            // instantiating the new Logger object. This prevents
            // other threads from performing the same block at the
            // same time.
            lock(mLock)
            {
               // Two or more threads might have found a null
               // mLogger and are therefore trying to create a
               // new one. One thread will get to lock first, and
               // the other one will wait until mLock is released.
               // Once the second thread can get through, mLogger
               // will have already been instantiated by the first
               // thread so test the variable again.
               if (mLogger == null)
               {
                  mLogger = new Logger();
               }
            }
         }
         return mLogger;
      }
   }

   // the constructor. usually public, this time it is private to ensure
   // no one except this class can use it.
   private Logger()
   {
      mLock = new object();
   }
}

public class SomeWorkerClass
{
   // any class wanting to use the Logger just has to create a
   // Logger object by pointing it to the one and only Logger
   // instance.
   private Logger mLogger = Logger.Instance;
}

The Observer pattern

In layman's terms again ...

This is where things get a bit more complex, but a bit more fun as well. The observer pattern is simply a mechanism where one object (such as our police officer from the example above) has the ability to dispatch a message to one or more observers. What they do with this message is completely up to them, and in fact, the officer doesn't even know who they are or what they do. He just knows how many observers there are, and has a predefined method for giving them this information.

Going back to the traffic example: whenever the officer needs to let traffic on one road stop moving, he shows his open palm to that road's direction. He is in effect sending all traffic, which is observing him for input, a message. The traffic facing him will interpret this message and stop, and all other traffic will interpret the message and subsequently ignore it has no effect on them.

Next, he waives to traffic in another road indicating that the drivers should start moving. Again, he has sent another message, and all the drivers will receive it in the same way, but act on it in different ways. Those facing him will start moving, and all others will simply ignore the message.
So far, we have one observed subject (the officer) and several observers (the drivers). The beauty of this pattern, however, is that there could be observers of other types as well, and as long as they can receive the officer's messages in the same way as the drivers (i.e. as long as they have eye sight and are looking at him), they can act on the messages as well. A simple example would be for the officer to actually be a cadet on training, and a senior officer to be sitting in his car watching this cadet and taking notes of the stop and go messages. Again, the senior officer is observing the cadet and receiving his messages in the same way as the drivers, but is acting on them in a different manner.

In programming terms, the officer would be sending his messages to the observers via an interface. If a given object implements an interface, any other object can interact with it through the properties and methods exposed in that interface without even knowing what the true nature of the object really is.

Putting this in code as well ...

The first part of this pattern is the use of an interface which will allow the Logger to dispatch new log entries to the objects observing it. The beauty of this is that those objects could b of literally any type and offer all sorts of functionality. They could be forms, file writers, database writers, etc. But the Logger will only know that they implement this interface and will be able to communicate with them through it.

interface ILogger
{
   void ProcessLogMessage(string logMessage);
}

Next is the management of these observers within Logger. We need an array (or List<>) to store them (technically speaking, though, we're only storing a reference to them), and a public method by which they can be added to the array:

class Logger
{
   private List mObservers;
   private Logger()
   {
      mObservers = new List();
   }
   public void RegisterObserver(ILogger observer)
   {
      if (!mObservers.Contains(observer))
      {
         mObservers.Add (observer);
      }
   }
}

Whenever we want to implement a new observer, we just need to ensure it implements the ILogger interface and does something meaningful when the ProcessLogMessage method is executed. An example would be a FileLogger object that writes the log messages to a file:

class FileLogger: ILogger
{
   private string mFileName;
   private StreamWriter mLogFile;
   public string FileName
   {
      get
      {
         return mFileName;
      }
   }
   public FileLogger(string fileName)
   {
      mFileName = fileName;
   }
   public void Init()
   {
      mLogFile = new StreamWriter(mFileName);
   }
   public void Terminate()
   {
      mLogFile.Close();
   }
   public void ProcessLogMessage(string logMessage)
   {
      // FileLogger implements the ProcessLogMessage method by
      // writing the incoming message to a file.
      mLogFile.WriteLine(logMessage);
   }
}

This class would then be instantiated and passed to Logger as follows:

public partial class Form1 : Form
{
   // the main Logger object private Logger mLogger;
   // a logger observer that will write the log entries to a file
   private FileLogger mFileLogger;
   private void Form1_Load(object sender, EventArgs e)
   {
      // instantiate the logger
      mLogger = Logger.Instance;
      // instantiate the log observer that will write to disk
      mFileLogger = new FileLogger(@"c:\temp\log.txt" );
      mFileLogger.Init();
      // Register mFileLogger as a Logger
      observer. mLogger.RegisterObserver(mFileLogger);
   }
   private void Form1_FormClosing(object sender, FormClosingEventArgs e)
   {
      // The application is shutting down, so ensure the file
      // logger closes the file it's been writing to.
      mFileLogger.Terminate();
   }
}

Putting the patterns together

Our logger will use both of these patterns, and the attached sample code uses two observers: the FileLogger mentioned above, and the actual form itself. The FileLogger logs the messages to a file, and the form shows the messages in a textbox control. This is obviously a simple implementation, but it could be used for much more creative purposes. We could have an observer that writes the log entries to a database table, another that concatenates all the entries and the emails the log, etc.

Whenever the form needs to have something logged, it simply executes the AddLogMessage() on the logger and passes it the log entry:

class Logger
{
   public void AddLogMessage(string message)
   {
      // Apply some basic formatting like the current timestamp
      string formattedMessage = string.Format("{0} - {1}", DateTime.Now.ToString(), message);
      foreach (ILogger observer in mObservers)
      {
         observer.ProcessLogMessage(formattedMessage);
      }
   }
}
public partial class Form1 : Form
{
   private void button1_Click(object sender, EventArgs e)
   {
      mLogger.AddLogMessage("The button was clicked.");
   }
}

Using the code

The sample project does not perform any complex operations, but showcases the logging discussed in this article. The main form has a button that, when clicked, increments a private counter and displays the value in a text box.

Every time the counter is increased, however, the logger is informed. In turn, the logger formats the message it receives and dispatches it to all the observers for processing.

As a nice plus, the Logger's AddLogMessage() method is overwritten to accept an exception. If the application throws an exception and we want it properly logged, we just pass the exception to the Logger and it extracts all the messages from the exception and inner exceptions (if applicable), puts them together, adds the stack trace, and then logs the whole thing. Very useful.
In conclusion

I hope this article and attached sample prove useful. If you have any suggestions for improvement I'm all ears, so post a message and let us know what you think!

A sample project is available for testing here: http://www.codeproject.com/useritems/ObserverLogging.asp

So I'm joining the crowds and blogging...

Never thought I'd be one to just follow the crowds, but I guess some things just make sense. Blogging sounds like an easy way to write and reach the masses easily, so here I am, doing just that.

Anyway, most of my entries will be programming-oriented. I do a lot of research on the net to figure out how to solve complex issues, and once I have it all working, I sometimes write it up in an easy to read article that attempts to spare the reader from the pain and hard work involved in doing all the research. I know I'd like to find more articles like this, so I'm sure there are others like me out there.

So here goes. I'll be posting my first article really soon.