Tuesday, February 04, 2014

History of the Development of our Plane Frame Analysis Application

The earliest dated Pascal source files I can find on our systems are dated 1990, however I didn't start using the program until the 1996 version. The 1996 version uses Turbo Gadgets to provide walking menus, and it otherwise uses the full graphics screen to display moment diagrams etc... This I wrote about in a previous post on my blog: since its a DOS based program using the full graphics screen its not overly compatible with windows: works Windows XP but not windows 7.

Back in 1996 the program was the only frame analysis program Roy Harrison & Associates (Now MiScion Pty Ltd) used. A few years later we purchased a MicroStran license, and a few years after that, rather than get a 2nd Microstran license we got a Multiframe license as a comparative check. Once we had the commercial software, this simple plane frame program fell out off use. {The application at this time had names alternating from f_wrk.exe to frame.exe}

Over the years however we have been repeatedly asked if we can supply software to help our clients sales personnel customise their structural products at point-of-sale (PoS). None however were really serious about investing in the development of such software: just an hopeful expectation that since we did everything by computer and we wrote the tools for such, that we could just throw something together. Not that simple.

The program as released here (cpframe), is the console(c) version of plane frame (pframe), I have removed the user interface. I never really used the user interface of the original program: this is because I wrote and used Borland Quattro Pro (QPro) macros to generate the data files. Our original projects involved standard calculations for sheds and carports, subject to a set of conditions but with a requirement to find the maximum height possible. Since wind loading is dependent on height, I needed to calculate new wind loads for each height until I found the limits of the structural section being used.

I could have wrote additional Pascal code and merged with pframe to achieve such objective, however already had QPro spreadsheets for general wind loading calculations, it was therefore faster to write QPro macros to generate the data files for pframe.

Similarly could have wrote more Pascal code to integrate coldformed steel design AS1538/AS4600 into pframe, but once again already had QPro spreadsheets for general design of members and connections. In terms of solving clients current problems had no time to develop a full application in Pascal.

Now whilst QPro could launch pframe, pframe didn't support command line parameters. Therefore I had to manually open the data file and run the analysis, then import results in QPro. Since pframe only supports a single loadcase, I had to repeat this procedure for each loadcase. Pframe was thus a bottleneck in the whole process that I wanted to remove. {This bottleneck was slightly improved once we got MicroStran and I modified the QPro macros to generate microstran .arc files. Microstran could deal with all loadcases and envelope the extreme values, thus reducing the total number of steps compared to using pframe.}

Back in 1996 however I had an aversion to touching the Pascal source code for pframe written by my father and adding the desired command line parameters or otherwise expanding the program. The aversion stemming from not having any simple and rapid means of testing the program to ensure I hadn't messed up the calculations.

So I needed an alternative method. Whilst moment distribution is easy enough to learn for continuous beams, its an otherwise cumbersome approach for plane frames (though extremely practical when there isn't anything else). American texts tend to show moment distribution calculations littered about a diagram of the frame whilst others present a more tabular approach. Either approach seems messy, and prone to error. Since I had a steel designers manual with Kleinlogel formula for the frames I was most interested in, I adopted Kleinlogel formula.

Still using Kleinlogel formlua is time consuming and prone to error, so I set up a spreadsheet to do the calculations. Spreadsheets being slightly cumbersome however, I decided to also program the calculations in Delphi 3 (Pascal).

Qpro macros for AS1170.2 wind loading
Parallel to this was the desire to expand the wind loading calculations and make them more complete and not just limited to the few scenarios most often encountered and manually handling other scenarios. Since more familiar working with arrays in high level languages like Fortran, Pascal and C it seemed easier to develop the wind loading functions  in Delphi (Pascal) than in the QPro macro language. So wind loading calculations were thus developed in parallel in QPro and Delphi, one used as a check against the other.
Now the problem with QPro macros is that the calculations are not up to date unless the macros had been run, and these were either run when the dialogue boxes collecting data were closed or when an hot key was used. This made the spreadsheets slightly cumbersome, but I had read somewhere that there was potential to create a dynamic link library (DLL) and add functions to QPro and this seemed possible using Delphi. Though I had't read in detail how to do so, and it seemed complex anyway, the potential was there. Hence the parallel developments in Delphi and QPro were not considered wasteful as they were expected to merge at some point in the future.

However, whilst wandering around book stores during my lunch break whilst working on contract in the city, I bumped into a book on Programming Office 97 using Visual Basic for Applications (vba). I had read some articles in computer magazines about Excel/vba but wasn't sure how vba related to Excel, plus I had a bias towards Borland software. Still I bought the book.

I had been hoping that Borland would merge the IDE's for Turbo Pascal, Turbo C and Turbo Basic and ensure they had the same function libraries, had built in language converters, including converters to turn GWBASIC into simple QPro spreadsheets or Paradox applications, and further more would be the programming languages for Paradox and QPro. They kind of did this, they threw Paradox and QPro away and developed Delphi and C++ Builder. Corel QPro I didn't like it was buggy. However, it was our 2nd QPro for windows license, and time was being wasted modifying my Borland spreadsheets to work in the Corel version. I didn't want to solve that problem by getting an additional Corel license. I was looking for an alternative spreadsheet, after reading the book on vba and office 97, I went and bought Office 97 Professional and the systems developer kit (SDK).

The wind loading functions I had programmed in Delphi, I translated into vba, thus making them available as functions in the Excel worksheet. A simple change to a spreadsheet cell and all the wind calculations upto date and the frame analysis using Kleinlogel also completed. Manually changing parameters in the spreadsheet I could quickly map out the potential of all the available c-sections for use in cold-formed steel sheds.

But still had a problem. Translating the dialogue boxes from QPro to Excel 97 wasn't so easy. Connecting the dialogue boxes to cells in Excel seemed cumbersome compared to QPro, I may have been doing it wrong but I had no real references for such. I tried the QPro way and that didn't seem to work: a similar approach does work in Excel 2003. Though there is still an issue of being able to abandon the dialogue box and not automatically updating the worksheet: such was not a problem with QPro1. Besides it appearing cumbersome to allocate data to drop down lists on the dialogue boxes, there was another problem with Excel and that was timing. There seemed to be a timing problem between getting data from dialogue boxes, evaluating user defined functions (UDF) and updating worksheet calculations. Either crashing or simply not calculating the correct answers.

Initially I had tried to replicate the QPro like wind loading macro's making use of the worksheets to store the data tables, but that appeared to be part of the timing problem and therefore I decided to abandon that approach in favour of using arrays in vba. Due to the problems with dialogue boxes, I abandoned them in favour of setting up input forms fully in the worksheet. Once the scope and life times of vba variables were better understood the workbooks worked fine.

But due to the problems encountered with programming vba, development continued in parallel in Delphi. I did attempt to iteratively change the structure height in an Excel worksheet and capture the resultant maximum frame moment and tabulate them. But a clear timing problem in that situation: the height can be changed faster than the worksheet can update the calculations. Incorporating delays could have probably fixed it, but why incorporate delays when objective to get calculated information as quickly as possible. Hence the Delphi program was expanded to iterate through the heights, or through the spans or through both heights and spans and calculate a table of maximum moments.

I then decided to produce a height/span chart and charting in Excel seemed easier than using Delphi graphics. Using Excel I could produce and store tables and charts and any other reporting may want, further more I could control Excel from Delphi. Unfortunately the Excel type library didn't import properly into Delphi due to keyword conflicts. The consequence was that programming Excel from Delphi was being done blind. Bit of a problem as Delphi uses [] for arrays and () for function parameters whilst vba uses () for everything. So that part of the application also needed parallel developments in Excel and Delphi, test what I needed to do in Excel then translate to Delphi.

This however was interrupted by changes to the wind loading code (AS1170.2), and since developing the application was a side line to day to day structural design, it was more important to update the general purpose wind loading Excel workbooks rather than update wind loading in Delphi. As a consequence Delphi was abandoned for developing the height/span charts, and it was all written in Excel/vba, all calculations in vba and thus avoiding problems of timing with worksheet calculation up dates.

Since we had been going down the path of developing a stand alone application in Delphi, pframe was part converted to Delphi to create a windows application with the graphics for the moment diagrams added, but without the rest of the interface developed.

However due to all the member design (AS4600/AS4100/AS1720) all being written in Excel/vba only, and the wind loading up date issue, it was considered that move over to development in Visual Basic might be more productive. So we obtained Visual Studio (VS) 2003 and VB.net, and then to get a second license we ended up with VS 2005.

But spreadsheets are still easier to format reports in, than messing around programming Delphi or VB.net. Sure those who prefer MathCAD type presentations think otherwise: that Excel is poor for presentation. For some reason there was some resistance to pushing forward with Delphi or VB.net development because of resistance to plain ASCII text files (.txt) or rich text files (.rtf), and complications of print preview and getting things to printers. But none of that is really a problem with MS Windows as notepad(.txt) and wordpad(.rtf) are available on each machine. Sure there is a possibility that the user can modify the results: but they would have difficulty proving and replicating such contended error in the program. Further today results are typically sent to pdf files rather than paper print out: and the pdf files can be edited.

Which is another point we had started to trial, generating pdf files, and produce all electronic documents early in the 1990's, but it was cumbersome to use pdf files where we needed results for further calculations. It was far easier to use paper printouts and mark up the required data. Scanning handwritten calculations to incoporate with computer generated calculations also produced massive pdf files, and so electronic documents were abandoned, we didn't have large enough hard disks to store the stuff, and zip disks were expensive. Hence further reason to integrate all the calculations electronically: eliminate the paper print outs for reference and produce smaller pdf files.

VB.net turned out to be significantly different from vba, and therefore it was put aside and pframe was converted to Excel/vba (xlFrame), so that it could interact more directly with Excel worksheet for input data and reporting.

Not long after doing that we were approached to provide a structures module for a carport/verandah spreadsheet. Whilst the structures module is relatively small the spreadsheet itself is relatively large, much of it is data and could probably be better done using MS Access: which is another development track pursued along with Paradox with respect to the materials management.

Now the Delphi application besides using Kleinlogel formula to generate height/span moment tables, it also generates data files for pframe, MicroStran (.arc) as well as AutoCAD LT scripts (.scr), it can also read data files from various other programs written. Much of this has been converted over to Excel/vba and extended further but as separate Excel workbooks. Attempting to gather a lot more vba code together into a single integrated applicataion hit some limit of Excel/vba. Whilst the code seems to run ok, its not possible to work on the modules as attempt to close/save the file it hits a memory limit and basically crashes. It won't save the file except through its recovery feature with all the vba code removed.

Since I don't consider I should use a better computer with more memory, nor that I should reduce the number of vba modules, further development in vba has stalled. Leading me to revisit VB.net.

The expectation was that could simply change the xlFrame into a COM automation object or .net component or similar, which can be plugged into Excel. Then all the parallel developments would disappear as all my wind loading and member design function libraries and the plane frame analysis could all be in vb.net and possibly a single library. Unfortunately that prior problem of the differences between vb.net and vba makes such conversion difficult for the plane frame analysis: though a simple conversion for the function libraries.

Also I want software like the carport/verandah software to export data files compatible with pframe and also to generate MicroStran arc files. When testing the data files exported by xlFrame they were not compatible with the 1996 version of pframe. This led me back to looking for Turbo Pascal source code to compile the original 1996 version of the program, and trace why the new data files were not compatible. Finding source code which compiled and used the same data files as the operational exe file, and which also produced correct results wasn't so easy.

The change in the file format was attributed to a change in how partial loads were defined. The error in the calculations was tracked down to dynamically allocated variables being freed from memory before results stored in those variables was actually used.

So having gone back to Turbo Pascal and given I prefer object pascal compared to vba, especially with respect to arrays in classes, it does seem that further development in Pascal may be the better option than vb.net, with Lazarus being a viable alternative to Delphi. Though a COM automation server may not be so easy to develop in Lazarus as it is in Visual Studio.

Anycase at the moment maintaining parallel developments in Turbo Pascal, Delphi 3, Lazarus, vb.net (VS 2005) and vba as I convert the record data structures into class/objects. The main difference at the moment is that an array has been converted into a collection in vba, though I may convert that back into an array and write property functions to access it. Both these vba approaches seem cumbersome compared to the other languages.