System failure

rakshithb174

Hello,

During loading of a RAPID module, the controller suddenly crashed and went into a system failure mode. I tried warm restart, but it didn't work. I tried the option "revert to last auto saved state" under advanced restart. This also didn't work. 

While the controller boots up, I can't access any functions such as Backup & restore or program editor on my teach pendant. I get the error ( see attached image).

I have a system backup from a week ago. I have made some important changes since then. Ideally I would like to restore the system with latest changes intact. Worst case option would be a system restore from backup. I really don't want to do a system or RAPID reset

How can i solve this? Also, since I can't access backup/restore option on teach pendant, How can I restore the system? 

graemepaulin

The shown error is pointing to an incorrect I/O configuration or Task configuration - did you alter either of these?

rakshithb174

graemepaulin said:

The shown error is pointing to an incorrect I/O configuration or Task configuration - did you alter either of these?

Nope. Everything was just fine until I started loading a new RAPID module. I didn't change any task or I/O config. 

I traced the sequence of events leading to system failure in robotstudio. The first error which caused system failure seems to be the load error. Following is the description of error message:

1. Load Error

40325: Load error

Description

 Task T_ROB1: A big enough free program memory block is not available. The ordered operation could not be completed.

Causes

The program memory is full or fragmented.

Actions

Check if large data structures could be split into smaller blocks. Use of installed modules can save program memory.

After this, I am seeing a bunch of domino effect error messages: 10048: Background task did stop, 50235: No Motion interrupts received, 50430: Underrun in the Axis computer, 39520: Communication lost with Drive Module, etc.

My hunch that loading new RAPID module caused failure seems to be correct. I am suspecting the system failure might be due to size of the RAPID module i was trying to load. Is there a restriction on number of lines per RAPID module or size? The module i am trying to load has about 30,100 lines.

I have 4 such modules. I load and unload them sequentially from a main program using StartLoad and WaitLoad instructions. 

graemepaulin

It sounds like you have run out of RAM - you can check how much is in use via the main menu item system info-> software resources->Rapid->RAPID memory.

Can you via RobotStudio see if the module you were loading is still loaded and if so delete it?

rakshithb174

I finally ended up doing a RAPID reset and restored the system using backup. The system is working now but i will have to manually update my RAPID files to include recent changes that i made this week

I have a total RAPID memory of 39.5 MB and current free memory of 35 MB. The file i was trying to load was roughly 6 MB in size with around 30,100 lines. To avoid this in future, Is there a restriction on number of lines per RAPID module or size? How should I handle large programs? 

graemepaulin

I would need to do some checking on that - but I think the limitation is the memory.
Also the file size of the RAPID module as stored on Windows is not the same as when loaded - again will have to do some digging to see how much difference there is.

@lemster68 can you help here?

graemepaulin

Found this:

All RAPID programs stored on PC or controller have ASCII format. At loading of RAPID program from C/controller memory into the program memory (internal format), the storage of the program needs about four times more memory space.

graemepaulin

Also found on a support case that when loading modules from the controller mass memory the module size limit is determined by the amount program memory available.

lemster68

Years ago I had a program with about 9500 lines loaded into an S4c+ controller without problems, biggest one I had ever done.  Its file size was 345k.  Even 6mb file x 4 is 24mb, so you had 35 available, which should have been adequate.  I  would break it down into smaller modules, anything that is common you should make a system module which should stay loaded always.  Are you storing and loading from the HOME directory?

DenisFR

Hello,

You can't guess memory used by program lines.

Like one comment line take less memory than one big area of big structured values.

rakshithb174

I think the numbers I mentioned above were after RAPID reset and restore. 

Yes, I am storing files in home directory and loading them sequentially from a main program using load, startload and waitload instructions. I am using unload switch in waitload instruction to unload and waitload at the same time. The main program is always loaded into memory.

I am guessing storing files in home directory might have taken up space leaving not enough for loading a module. 

Is there a way to store modules externally and load them automatically from that location?

Is there anything we can do from code side to check available free memory before load? 

lemster68

FileSize function can tell you the size of a file.  FSSIze function can tell you the size of the file system where something is stored.  I am not sure that the query can be directed towards the actual operating RAM of the controller.

rakshithb174

What is the total size of controller mass memory and program memory for IRC5 controller?

rakshithb174

Also, in my code, First I load a module stored in Home directory of controller using Load instruction. After load is complete, I also start to load the second module file, also stored in Home directory using StartLoad instruction. 

Is there a limit on module size during this scenario or does it depend on available prigram memory in controller? Both files are of same size (~6 MB with 31k lined of code). 

So, I am guessing that I should atleast have 6+6 = 12 MB of free program memory to successfully finish the load right?