Troubleshooting and Debug

The first thing a lot of engineers and technicians do when trying to isolate a problem on an automated machine is go get their laptop and use programming software to see where the issue might lie. In some cases this could be the best solution, but in many cases there are much simpler ways.

I received a call a few months ago on a system I had put into a plant in about 2001. The customer was in a panic because the machine was down and they had “lost” the program. They also said the program had stopped working so they would need someone to come re-download the program for them. This customer was about four hours away and I was involved in another project that I couldn’t leave, so I spent some time trying to analyze the problem from a distance.

Q. “I’m quite sure I left a backup of the software with someone there, have you tried going online?”
A. “Well, there is a disk here with a bunch of files on it, but they must be corrupted or something. We can’t open the files.”
The system was an Automation Direct (Koyo) PLC controlled material handling line. The reason that platform had been chosen was that the company had very little automation in their plant, no specifications and a very low budget. Such a low budget, in fact that they had never purchased the software. Of course without having the software you can’t open the files, they would look like gobbletygook to any other program.

Q. “How do you know the processor has stopped working?”
A. “Well, when we push the buttons on the touchscreen, even in manual mode, nothing happens.”
The fact that the machine could be put into manual mode from the touchscreen should provide a clue. If the pushbuttons toggle on the screen and indicators change color, the program is running. At least in programs I have written. The mode status is reflected by the indicator on the screen, which only changes state when the program is running. Another note here: sometimes people without experience in automation may say “that part of the program isn’t working” or “the program has changed”…. programs either work or they don’t, and they never change themselves. Look elsewhere for the problem.

In this case I suggested taking a look at the status of the input and output cards. Since nothing seemed to work and no faults were being indicated I suspected an output problem. This is pretty easy to diagnose; If a machine is in manual or maintenance mode and you press a button, the corresponding output should turn on unless a “permissive” bit is preventing it. Since this was a material handling application there were some permissives preventing material from being dropped on the floor, but most of them had warnings or faults associated with them that would display a message under certain conditions. Many simple actions such as turning a conveyor on and off had no permissives in manual mode since the product being moved was 1. very slow and 2. sometimes offloaded by hand with the conveyor running. As I mentioned, a very low-budget, simple application.

The first step in diagnosing output problems is to look at the light on the output card and see if it comes on when it is supposed to. For simple manual applications this is pretty much a simple cause and effect relationship for many actuators and motors. The second step, if the output light is on, use a multimeter to determine if there is voltage at the output terminal. If there is, see if the voltage is present at the actuator terminal. Fuses are often placed between the output and the device, so check that also. If the voltage is present at the actuator the problem may be with the actuator itself; maybe a bad valve, burnt input on a drive, bad coil on a starter, loose wire, lots of possibilities. A good electrician or technician is usually able to easily pinpoint the problem on a single actuator. Input troubleshooting is much the same; if a sensor should be changing state to cause some action, first simply look at the indicator light on the sensor. In the case of a photoeye, there is also often a power light. If the sensor indicator changes state, take a look at the point on the input card. If it also changes state, the problem may be something in the programming not allowing an operation to take place, but most machines either use a button indicator to reflect the status of an input or actually map all inputs to an I/O screen for diagnostics.

In the case of this customer, either the output card itself was bad or the voltage was not present on the card’s power terminal. Since the customer didn’t call me back I never did find out what the problem was, I have to assume they got the system back up and running though.

Debug can be an entirely different story. Since the program hasn’t been proven there is often a need to go online with the processor for troubleshooting. Sensors may be wired to incorrect terminals or mapped wrong in the program, valves may be plumbed backwards, wired incorrectly or mis-assigned in software. All part of typical startups despite the best intentions of designers, programmers and technicians. During startup and debug a laptop or computer is usually online with the processor anyway, which makes it a useful and quick diagnostic tool.

Don’t discount simple troubleshooting and debug techniques just because you can’t go online with the processor though. There is no substitute for good old common sense and logic!