Building Out Distributed Apps (Big Data)

Yesterday, I attended a webinar by O’Reilly on how to reduce the pain of building out distributed applications. The focus was on scalability, which makes sense, since this is why you would want to distribute your applications.

Apart from the host’s unfortunate resemblance to Little Lord Fauntleroy, there was some interesting observations to be made. To wit:

Engineers versus Ops

When there’s an issue affecting your customer in large systems, it is most likely an engineering issue, especially in emerging products. You need to staff up on Engineering talent for your projects at a much greater rate than Ops.

Data is not always relational

Data these days is more than OLAP stuff. Things being captured and crunched include data graphs, key-value pairs, etc. So, something non-SQL based might be called for as a datastore. Only a handful of SQL features are used in most large data projects. As the data sets get larger, SQL gets less useful.

Real-time versus Batch Processing

Something to consider. How is your data being created, in one-sy/two-sy fashion online, or in large grabs of data. This will affect your basic understructure.

Cost of Research

It is very easy to under-estimate the cost of research when moving into a new area. Executive management wants hard numbers to be able to plan and manage costs, but anybody who’s developed new systems knows that costs tend to be unpredictable because you just don’t know what you don’t know yet.

What is your experience involving Big Data and Distributed Applications?

Some Do’s and Don’ts for Twitter from Business Insider

Go to this link, read the items, then check out the other links…

http://www.businessinsider.com/6-dos-and-donts-for-your-companys-twitter-account-2010-3

  

LiquidOffice and Printing PDFs Using JavaScript

I recently helped a client work through a particularly tricky issue with a Liquid Office PDF form.  Normally a form is completed by an end user, possibly processed through a workflow and  finally released into an EDMS system with the help of a Connect Agent (for instance, ImageSource’s ILINX Connect for eForms product).  However in this case, the form is simply used for printing.

The form’s field elements consist of a drop down with a list of titles, text boxes, barcodes, and a big shiny print button.  When the user presses the print button, the form goes through each title defined in the drop down list, sets the few bar-code values based on the text boxes values, then calls the JavaScript print method.  The first print request displays the standard Print Dialog, and the rest print silently.  To print one batch of cover sheets, print is called ten times.  Using Form Designer’s preview functionality yields the expected results.  However, when the form is published to the Liquid Office server and the form launched from the Web Desktop, the first cover sheet is never printed.  Instead, two copies of the second cover page are.

So what is going wrong?  After judicious use of the JavaScript alert method, I was able to determine that there is a bug when scripting a print dialog while Adobe Reader is hosted inside of a browser window.  When this.print(true) is invoked, Adobe Reader continues executing any remaining scripts until the dialog is displayed or it reaches a statement that blocks script execution, like a message box or another print statement.  I tested multiple versions of Adobe (7,8 & 9) and several different browsers (IE, FireFox, Safari), but with each combination ran into the same bug.  A side effect of this bug, is that if you assign a variable to the return value of print, that value is always true, even if the print job was canceled!

To get around this bug, I took a page from the AJAX playbook.  Since I needed a way to block script execution until the print dialog was displayed, I used the JavaScript setTimeOut method.  If you don’t know what setTimeOut does, it invokes a function after a specified number of milliseconds.  You can find a more complete explanation here.  Once the print dialog has been displayed to the user, the Print Properties object must be set and passed into each successive print request.  Otherwise the settings selected by the user from the dialog will be ignored.  Here is the JavaScript code that makes the solution work.

There are a number of tricks I tried that did not work.  For example, I wrote a function to pause execution by looping for a specified amount of time, but that didn’t work.  JavaScript also does not have a built in method to put the current thread to sleep.  A message box after the first print statement would have done the trick, but would have been confusing for the user.

Tyson Magney
Senior Developer
ImageSource, Inc.

  

ECM Best Practices: Unit Testing and NUnit

Everyone agrees that testing is a good thing. Not everyone agrees on how much testing is cost-effective and what exactly the kind of testing is right for a specific piece of software or product. But automated unit tests cam help you ensure your software is healthy.

Unit tests operate on the smallest possible section of code, on logical modules. In software design, a program of any decent size will be broken down into modules. These modules have inputs, outputs and behaviors. These inputs and outputs and behaviors are used by other parts of the program to perform work and can be monitored or tested by the test software.

Often, a software module is deeply bound up inside the containing framework and it is not obvious how to test a subset of the code in isolation. (Testing multiple modules together is referred to as “integration testing” and will be discussed in a later post). Fortunately, there are software constructs available to mimic or “mock up” objects which the module under test needs.

A case in point is a product we are revising called ILINX Release Script 9.0. The ILINX Release Script is a module which is run mainly inside the process space of Kofax’s Ascent Capture product. There are a number of complex environmental elements which supply runtime context for the various ILINX components.

Because we are developing in C#, and are using the NUnit testing tool, we are able to make use of the NUnit’s Mock objects to help in mocking up objects to test against.

By stubbing out the functionality of the execution context, we are able to exercise specific portions (or units) of code with very specific and controlled data and confirm the software’s behavior and output. If you are using .NET, I encourage you to look into NUnit (http://www.nunit.org/index.php) it’s ability to generate Mock objects.

  

EHLLAPI – The Olde Way to Integrate

I came across this title on one of Inventu’s pages when researching an integration issue for a customer…I must admit, it made me smile, and if I LOL’d, I would’ve…