Monthly Archives: December 2013

Load XML File into Associative Array in PHP

Here’s a neat bit of code to load an XML file into an associative array in PHP. Note that if an XML field is empty it creates a zero element array for that field (for some reason) so I’m checking for this and catching it.

$xml_obj=simplexml_load_file($filename);
$json=json_encode($xml_obj);
$this->file_contents=json_decode($json,true);
			
foreach ($this->file_contents as $key=>$value)
{
  if (is_array($value))
  {
    $value='';
    $this->file_contents[$key]='';
  }
  echo "field::$key value:: ".$value."<br />";
}		

C# MD5 File Hash and PHP Comparison

Further to my recent formless multi-threaded C# application blog post I’ve just implemented a simple MD5 hashing system to ensure that files are uploaded without the contents being changed. I’ve created a new method in my File Uploader class to calculate the MD5 hash of the files I’m uploading. You wouldn’t want to do this on the fly for larger files because of the time taken but the XML files I’m dealing with are less than 2kb in size so it’s not an issue. Here’s the MD5 method for my uploader class.

        private string md5Hash
        {
            get
            {
                using (var md5 = MD5.Create())
                {
                    using (var stream = File.OpenRead(WebExtensionsService.XMLFolder + "\\" + this.Filename))
                    {
                        byte[] md5_bytes=md5.ComputeHash(stream);
                        return BitConverter.ToString(md5_bytes).Replace("-","");
                    }
                }
            }
        }

Note that you need to convert the byte stream to a string and you’re going to have to strip out all the “-” characters to make the MD5 string look normal.

Once I had the MD5 hash I simply appended it to the URL I’m uploading the file to. Like this:

    byte[] byte_response = wClient.UploadFile(WebExtensionsService.UploadLocation+"?md5="+md5, "POST", WebExtensionsService.XMLFolder + "\\" + this.Filename);

The final step is to calculate the MD5 of the uploaded file in PHP and then compare this with the passed MD5 hash. I do this in the code below as well as a couple of other checks to make sure the file is of the correct type and isn’t too big.

	if (isset($_GET['md5']))
	{
		$md5=$_GET['md5'];
	}
	else
	{
		echo "Error : No md5 hash";
		die();		
	}

	if (substr($_FILES["file"]["name"],strlen($_FILES["file"]["name"])-4)=="xml")
	{
		echo "Error : Invalid file type";
		die();
	}
        if ($_FILES["file"]["size"]/1024>1000)
	{
		echo "Error : File too large";
		die();		
	}
	
	$tmp_name=$_FILES["file"]["tmp_name"];
	
	$calculated_md5=md5_file($tmp_name);
	
	if (strtoupper($md5)!=strtoupper($calculated_md5))
	{
		echo "Error : MD5 File Hash Mismatch";
		die();
	}

Help Brian Buy New Computers

Brian Lunduke has a cool blog that I read. He’s a Linux advocate, sometimes tech journalist, and writer of children’s and comic books. And sadly someone just robbed him and stole all of his computers. Rather than appeal for cash to get new ones he’s encouraging people to spread the word about his books (which can be bought here) in the hope of selling enough to pay for new computers. As a prize he’s going to randomly choose someone who helps spread the word to become a character in his next book entitled “Steve’s Laptop”. Now I don’t particularly care about winning anything but I would like a copy of “Linux is Badass” so I’m hoping that one of my readers will pony up the $2 to buy it for me. I’d do it myself but I am cheap.

C# Formless Application with Multi Threading

I’ve been having a bit of fun in the last few weeks developing the next major release of Time Clock MTS. Part of this release includes a system that sends out emails based on certain events occurring within the software. To accomplish this I could have included some POP email fields and asked users to enter their mail account details. Or I could have used MAPI email. But both of these approaches are ugly as they require input from the user and don’t work for web mail.

So I decided on an approach that goes something like this:

1. The existing application generates an XML file when an event email is required and saves it into the appropriate application data folder.
2. A process monitors the data folder and uploads the files to a web server as it finds them. Each upload process is executed on its’ own thread.
3. The web server processes the XML file and sends emails as required.

Step 1 was easy enough even in Visual Basic 6 (which Time Clock MTS is developed in). However step 2 was always going to be tough so I decided to build a small application in c# to perform this role. The VB6 application would start the C# app on startup and the C# app would shut itself down when the VB6 application closed. Step 3 would be accomplished easily enough with PHP. The key advantage (as I see it) with this approach is that the only requirement for a client computer is an internet connection. It doesn’t require an email client to be installed (like MAPI does) and doesn’t require the user to know and enter some arcane email server settings when they first install the program.

I won’t bore you with Step 1 or 3 but Step 2 had some interesting code that I thought I might share. The C# app is a standard Windows Forms app but I hide the form (except when the debugger is running). I could have made this a windows service but that seemed to introduce a level of complexity (and difficulty in debugging) that was unwarranted. Here’s how I hide the form in the form constructor:

            InitializeComponent();
            if (!Debugger.IsAttached)
            {
                // Prevent the window from showing up in the task bar AND when Alt-tabbing
                ShowInTaskbar = false;
                FormBorderStyle = FormBorderStyle.FixedToolWindow;

                // Move it off-screen
                StartPosition = FormStartPosition.Manual;
                Location = new Point(SystemInformation.VirtualScreen.Right + 10, SystemInformation.VirtualScreen.Bottom + 10);
                Size = new System.Drawing.Size(1, 1);
            }

The work that the form does is handled by a WebExtensionsService class that includes a couple of timers. The class is initialized in the form constructor and the timers are started at the same time. The first timer checks for the existence the parent VB6 program process. I’m sure there’s better ways of doing this but the method I used below works fairly well with little or no CPU overhead and raises an event.

        private void timCheckForClose_Elapsed(object sender, ElapsedEventArgs e)
        {
            //writeLog("timCheckForClose update");
            Process[] adminProcesses = Process.GetProcessesByName("appname");

            if (adminProcesses.Count() == 0)
            {
                if (_bAppRunning == true)
                {
                    _bAppRunning = false;
                    OnAppNotRunning(EventArgs.Empty);
                }
            }
            else
            {
                if (!_bAppRunning )
                {
                    _bAppRunning = true;
                    OnAppNotRunning(EventArgs.Empty);

                }

            }
            

        }

Then I could consume this event in the C# form with this:


        private void WebExtensionsService_AppStopped(object sender, EventArgs e)
        {
            Application.Exit();
        }

I used the Command Line Parser Library to read in a range of command line options that were passed to the C# application when it was started by the parent VB6 application. These settings were used by the second timer to do the real work of this C# forms app. Namely scanning a folder looking for XML files and then uploading them to a server. The second timer in the WebExtensionsService class does this:


        private void timCheckFiles_Elapsed(object sender, ElapsedEventArgs e)
        {
            this.checkForXMLFiles();
        }

        public void checkForXMLFiles()
        {
            Classes.Uploader uploader;

            if (uploaderThreads.Count < this.MaxThreads)
            {
                if (bDirectoryExists(this.XMLFolder))
                {
                    DirectoryInfo info = new DirectoryInfo(this.XMLFolder);
                    FileInfo[] files = info.GetFiles().OrderBy(p => p.CreationTime).ToArray();
                    writeLog("checkForXMLFile:: " + files.Count().ToString() + " files found");
                    foreach (FileInfo file in files)
                    {
                        if (!uploaderThreads.ContainsKey(file.Name))
                        {
                            uploader = new Classes.Uploader();
                            uploader.Filename = file.Name;
                            Thread workerThread = new Thread(uploader.DoWork);
                            uploaderThreads.Add(file.Name, workerThread);
                            workerThread.Start();
                            writeLog("Starting uploader thread " + uploaderThreads.Count + " of " + this.MaxThreads);
                        }
                        if (uploaderThreads.Count >= this.MaxThreads)
                        {
                            writeLog("checkForXMLFile:: MaxThreads " + this.MaxThreads.ToString() + " are busy");
                            break;
                        }
                    }

                }
                else
                {
                    writeLog("checkForXMLFile::"+this.XMLFolder+" doesn't exist");
                }
            }
            else
            {
                writeLog("checkForXMLFile:: MaxThreads " + this.MaxThreads.ToString() + " are busy");
            }
        }

The Uploader class is my worker class that is used to upload a file in it’s own thread. Note that I have a MaxThreads setting and maintain a collection of UploadedThreads to make sure I don’t try to upload the same file twice.

And for interests sake here’s the method from the Uploader class that does the actual work. It’s worth noting that the WebExtensionsService class is a singleton and contains all of the global settings I need. I’m sure this isn’t the most elegant way of doing things but it’s always worked extremely well for me in the past.

        private bool uploadFile()
        {
            bool bReturn = false;
            string sResponse = "No Response";
            WebClient wClient = new WebClient();


            try
            {
                byte[] byte_response = wClient.UploadFile(WebExtensionsService.UploadLocation, "POST", WebExtensionsService.XMLFolder + "\\" + this.Filename);
                if (byte_response != null)
                {
                    sResponse = System.Text.ASCIIEncoding.ASCII.GetString(byte_response);
                    WebExtensionsService.writeLog("Upload " + this.Filename + " Reponse :: " + sResponse);
                    if (sResponse == "1")
                    {
                        bReturn = true;
                    }
                }

            }
            catch (Exception e)
            {
                WebExtensionsService.writeError(e);
            }
            if (bReturn)
            {
                sResult += " UPLOAD SUCCESSFUL";
            }
            else
            {
                sResult += " UPLOAD FAILED RESPONSE " + sResponse;
            }
            

            return bReturn;

        }

Right now the uploadFile method waits for the correct response from the web server. I am thinking about not just uploading the file but also uploading a checksum of the file contents so that the web server can check that the file has not been corrupted in the upload process. But that’s something for another day.

The final step in the application is how to close it out cleanly. I showed earlier how the Application.exit() method was called when the parent VB6 program process was no longer present. Here’s the code that makes sure that all the uploader threads are closed out before the application exits.

        private static void OnApplicationExit(object sender, EventArgs e)
        {
            Classes.WebExtensionsService WebExtensionsService;
            WebExtensionsService = Classes.WebExtensionsService.Instance;
            WebExtensionsService.writeLog("Closing TimeClockMTSWebExtensions");

            foreach (KeyValuePair<string, System.Threading.Thread> entry in WebExtensionsService.UploaderThreads) //wait for uploaded threads to finish
            {
                System.Threading.Thread thread = entry.Value;
                thread.Join();
                WebExtensionsService.writeLog("Waiting for " + entry.Key + " uploader thread to finish");
            }
            WebExtensionsService.writeLog("Closed TimeClockMTSWebExtensions");
        }

The key here is the thread.Join() call. This blocks the main C# program thread until the child threads (stored in my UploaderThreads collection) have completed their work.

Doing this work in C# is far easier than trying to do the same thing in VB6. The C# network code is about a million times easier and more robust. Part of me would LOVE to port the entire VB6 application to .NET but there’s three things stopping me. Firstly, the sheer amount of work. We’re talking about 80,000 lines of code that need porting. Not a trivial task. Second, ADO code in .NET simply isn’t as good as it is in VB6. It’s slower, handles connection pooling very poorly in comparison with VB6, and file sharing of a simple database like MS Access just doesn’t work as well. In the past I’ve had to perform tricky pre-caching of ADO data in .NET to get it performing even remotely as quickly as VB6 code. And finally, it scares me witless mainly because of all the reasons discussed in this great article by Joel Spolsky.

That being said stepping back to VB6 to do development work instead of using C# is like owning a 2013 model car and being forced to drive a leaky 1969 Volkswagen Beetle every second day. It works but it’s not fun. As a result I’m trying to build new features and move older non database reliant features (such as web camera image capture and network time checking code) into COM visible .NET code. However, because of the problems with database handling in .NET I don’t think I’ll ever get rid of VB6 altogether unless Windows drops compatibility for it.

Take a Deep Breath

Sometimes being the place where all bucks end up stopping can be a little wearisome. I’ve been tail up working on a major release for one of my products for a few weeks now and this morning I was keen to get a large chunk of a new feature polished off. However, as I am responsible for both technical support and new product development (and everything else in my mISV) I try to get support issues resolved before I start development for the day. Usually this takes an hour or two and leaves me the majority of the day to work on development but this morning I just couldn’t seem to get (what I thought) was a fairly basic point across to a user of one of my products. The email chain went something like this:

To: Support
From: User

My employee can’t clock in.

From: Support
To: User

What product are you using? What version? What browser (and browser version) are you using. Are you getting any error messages when your employee tries to clock in?

To: Support
From: User

It tells them they are not authorized to log in from that computer and I have to manually enter their times.

From: Support
To: User

Oh, you’re using ProductNameRemoved. You’ve set up IP login restrictions to stop people from logging in from un-authorized devices (such as their cell phone). Just log into ProductNameRemoved as the administrator and go to this screen (URL REMOVED) and you’ll be able to adjust the allowed computer IP addresses for your account.

To: Support
From: User

We never set that up.

From: Support
To: User

I’ve just logged into your account and taken a look and you’ve restricted logging into your account to the following IP addresses (ADDRESSES REMOVED)

To: Support
From: User

Oh. Yes I did set that up. My employee can’t clock in.

From: Support
To: User

Why don’t you check what the IP address is on the computer the employee is trying to clock in from. Perhaps it has changed. Once you work that out you can adjust your restriction rules.

To: Support
From: User

Why did you change the computer IP address?

From: Support
To: User

We didn’t. Perhaps your internet service provider has changed your IP address? Unless you have a contract that specifies a fixed IP they can certainly change.

To: Support
From: User

I will check into that. I will give them a call. Thank you for your help!

By this time I’d wasted nearly an hour with this back and forth and (in the words of Homer Simpson) my urge to kill was rising. Thankfully my partner told me to take a deep breath and step away from the computer and go and do something else. So I ducked out, got some sushi, calmed down and have returned for an afternoon that is being entirely more productive.

Dipping my Toes in the ODesk Pond

I’ve known for a while that the main limiting resource in the on-going growth of my software business is me. I’ve had real issues being willing to farm out work to others and it hasn’t helped that when I have reached out and found others to do work for me that results have mostly been awful. There’s been one good experience on 99Designs a few years back and about six months ago I found a Virtual Assistant in Canberra who has been doing good work for me. But other than those two I’d say quite cheerfully that I’ve wasted my money paying people to do work for me.

My problem right now is that I want to build some Explainer Videos for one of my products and need some inspiration in terms of the tone, and content of the scripts for those videos. The videos will be used on Ad Words landing pages. I want them to be short (two minutes or so) and to quickly establish empathy with the potential users (we understand your problem) and then present them with a solution (my software). I’d like the videos to be light-hearted and, if possible, amusing. After all, my products are pretty boring so any humour that can be injected can only be a good thing. I’ve written one script myself but really want a few so I can get 10-15 minutes of voice talent recorded with the eventual aim of getting several videos made and A/B testing them. Now, I know that I should write the scripts myself but to be perfectly honest I don’t have the time or the talent to write multiple scripts from a blank page. So I was hoping that I could find some people with some experience creating explainer video scripts and they could write some scripts for me. Even if I don’t use the scripts at least they’d provide me with some inspiration so I could develop a few more myself.

ODesk seems to be the freelancer site of choice these days. So I’ve toddled off and set up an ad for 2 x 2 minute explainer video scripts. After a week or so I’d got 6 applications and a few hours ago I chose two of the applicants to interview via Skype. I’ve since conducted those interviews and hired both applicants. My expectations are not high but for the cost a few hundred dollars I might get one script I can use and I’m fairly certain I’ll get some useful ideas I can work on myself.

WordPress feeds my desires for useless statistics much better than MovableType ever could. Out of the box I can see how many words are in a post but even better I can get aggregate stats for the blog as a whole using a plugin like Word Stats. Below you can see the statistics as they stand right now. Pointless but still cool to know.

What Mark Did Statistics

What Mark Did Statistics

Cord Panorama WiFi Internet Radio Player

Cord Panorama Internet Radio Player

Cord Panorama Internet Radio Player

We’ve been wanting a wifi internet radio player we could listen to around the house and perhaps take outside, plug in, and listen to out there. So, after a week or two of looking around online (and not really knowing what we needed) I made a spur of the moment decision and above you can see our new Cord Panorama WiFi Internet Radio/DAB+ Digital Radio / Media Player / FM Radio. We just bought it at 40% off of normal price from JB Hifi at the insanely good price of $132. It can play 100,000+ internet radio channels, stream media from a DLNA server, is a DAB+ digital radio receiver, and is a regular FM receiver too. And to top it all off it has an auxiliary input so you can plug in your iPod or other device and play music from that too. It has a bit of a retro look wood finish on the sides and top and a silver front panel with a black and blue OLED display. There’s a single speaker with a 7W digital amplifier, which it turns out is more than loud enough to fill our living area with music or to put it under our verandah and hear it while we’re in the backyard. There’s also a little remote which you can see below.

Cord Panorama Remote

Cord Panorama Remote

The device can work with a WiFi or wired connection and has a very simple setup wizard that allows you to enter your WiFi password pretty easily. Once setup the radio is controlled from either the front panel or remote. It operates in one of several modes (FM, DAB+, Media Streaming, Internet Radio, or AUX) each mode has slightly different menus and it’s own set of user selectable favourite stations. DAB+ and FM tuning were fairly effortless and intuitive. The media streaming mode detected our media server without a hassle and browsed quickly and easily through our media library and played our music perfectly.

Internet radio proved to be a little more problematic as the Cord unit kept timing out when attempting to retrieve a channel list from the internet. It started working after a day or so and allowed us to choose from many different Australian internet radio stations by genre. We could also search through the complete radio list allowing us to find our favourite stations easily. I’m not sure what the problem was with the network timeouts but I do note that the Cord Panorama has some sort of online integration with Wifi Radio Frontier site. And that site was down yesterday and this morning so I bet the two events are linked. The online integration allows the Cord Panorama users to find internet stations online and mark them as favourites and then have those stations downloaded to the radio unit. Pretty neat.

So what do I think of the Cord Panorama Internet Radio Player? I think it’s perfectly fine. The user interface is easy to understand, I didn’t have any problems with it and my partner has been using the radio from the front panel without issue. The front panel display is clear and useful and the tactile feel of the buttons and dials are quite nice. I’m even fond of the retro wood-grain finish. If I had anything bad to say I’d restrict it to the remote which is a bit small and fiddly and probably not that useful. But putting that to one side the Cord functions as advertised and pretty much flawlessly. The sound quality is good and it plays music from a variety of different sources exactly as advertised. And at the price we got it at I consider it to be something of a bargain. Highly recommended if you’re in the market for a wireless internet radio player.

Update December 9 2013

When I was having problems with the player connecting to internet radio stations I shot off an email to the distributor of the Cord Panorama in Australia (Volition Australia) asking them if they knew what the problem was. To be honest I wasn’t expecting a response as I’ve never had much luck contacting service departments by email before. But here we are at 10:00AM on a Monday morning and I got a response! Amazing service. Here’s what they had to say:

Mark,

Thanks for your enquiry.

It seems that Vtuner, the internet radio service used by panorama was down for a period of time over the weekend.
After checking this morning, it is up and running.
Let me know if you still have any problems.

Regards,

Michael Noorbergen
Technician

So, there’s your answer. The Cord Panorama uses the Vtuner website to get some data and it was down on the weekend.

Here we are on WordPress

Well it’s taken me half a day but it was fairly painless. I’ve ported over about 80 blog entries to WordPress and put some 301 redirects in the .htaccess file to redirect the old MovableType URLs to the new ones. I had to install a number of WP plugins to get things working the way I wanted. This included AuthorSure (for Google+ authorship), Google XML Sitemaps, NK Google Analytics, Responsive Lightbox, WordPress SEO, and WP Syntax Highlighter. I must say being able to seamlessly upload images from within the WP editor and markup code directly (without having to remember codes) is fabulous.

There’s still a couple of things to sort out, mainly to do with the logo. But I am sick of doing this today so I’ll leave it for another (more motivated) day.

Moving this blog from MovableType to WordPress

Gentle reader, I am sure you’ll be pleased to know that this blog gets almost no traffic. It’s a busy day when one visitor shows up here to disturb the electronic cobwebs. This doesn’t bother me much but I’ve always wondered if putting the blog on a different platform would help. Right now it’s on a very old version of MovableType. I’m going to move it onto WordPress, which is the most popular blogging platform out there. I’ve used it enough to know it’s pretty freaking awesome. I’ve also heard that WP is the ducks testicles as far as SEO goes especially when a good SEO plugin like Yoast’s SEO is used. While I don’t particularly believe that a simple change of blog platform can help that much there’s no harm in testing. So as something of an experiment I’ve decided to move this blog to WordPress, install the Yoast SEO plugin, do a bunch of 301 redirects from the old blog entries to the new ones and see if the blog actually gets any improvement in terms of the traffic it sees. For reference here’s the traffic stats for the last 3 months (prior to December 6 2013).

Vists: 431
Unique Visitors: 320
Pageviews: 676
Pages/Visit: 1.57

The visits were sourced from referrals (263), direct type ins (97), organic searches (56) and social referrals (15).