Frontend Control with VAX and VME/PowerPC

Version 02.-Feb.-2004 / jn


Starting the VAX  Frontend Processes

The direct control of the frontend (i.e. the ULO synthesizers, the frontend control units and some other units) is done by a couple of tasks running in a PowerPC processor under VxWorks.  The  overall synchronization with the general control system (in the VAX) is made by the frontend control program in the VAX control computer. These two computers communicate on one side by TCP/IP socket connections and on the other side by synchronizing bits, which are exchanged by hardwired output and input modules in the CAMAC and the VME crate.

The frontend processes in the VAX (also serving the VME processor) are started by the well known symbol:

FEP

Using these symbol will always first stop all the frontend processes existent in the VAX. Then the frontend processes are started.

The frontend procedure VME_FRONTEND.COM – hidden under the symbol FEP - starts two processes in the VAX:

 VMESOCK:   is a socket server process (TCP/IP) to communicate with the PPC processors

 VMEFRONT:  is the frontend organizer process for the VAX control system using  the VME processor for the control of the frontend hardware.

If the PPC frontend processor is not yet running, it first has to be (re)booted before one can start the VAX  frontend processes with  @VME_FRONTEND or FEP (How to start the VME processor, see below !). If the PPC frontend processor is running one can directly start with the procedure.

The configuration phase of the frontend processes in the PowerPC will take some time, because it first checks the existence of all components and instruments defined in the startup file like frontend-control units, synthesizers and counters.

Messages and errors are displayed in the usual way on the VAX process display (CMON) and additive on the new frontend-control display and the new message-handler display.

 

The On-line Display for the Frontend


A new display for the frontend exists as part of the general data display program (vardispl) for Linux. The parameters are self describing.

 






VME Frontend Control Crate

The crate has the following components:

  • Crate Processor  (PowerPC,  MVME 3604,64MB memory) OS: VxWorks 5.5

    • with Transition Module at the backside (MVME 761)

    • with  Reflective Memory Board  VMIPMC  5576 XL (4 Mbyte) (PMC-module)

  • PCI-Carrier board for  4 Industry Packs  (IPPC 2636) , directly mounted on the MVME 3604 board

    •      with the following IPs:

    •  Slot A:  16 digital inputs, optically isolated, TIP600-10

    •  Slot B:

    •  Slot C:

    •  Slot D: IEEE 488 (GPIB)  Instrumentation Bus  (IP-488)

  • VMEbus-Signal and Time Generator  (TrueTime Modell 560-5605 VME-SG)  connected to IRIG B, which is delivered by a special station clock receiving GPS-satellites'.

  • VME-PROF-S ProfiBus-Interface (Dorsch Microsystems) connected to the ProfiBus-station in the control room







The VME crate, which is used for the frontend control. REBOOT points to the button, where the PowerPC processor can be restarted.

 

Hardware Connections between Control Computer and VME

The control computer transfers the information of the actual synchronisation state to the VME processor by setting certain bits in an output register 3074, which is hardwired to an input module (TIP600-10) in the VME crate. Other information is exchanged by TCP/IP socket connections via ethernet. The following bits are set by the control computer:

  • Init , Setup, Prepare, Measure, Cancel:  Are the normal synchronisation phases oft the control system for the Frontend.

  • Config (always together with Cancel):  Demands a restart of the configuration programs in the VME processor ( searching  for synthesizers, frequency meters and frontend control units). That normally is done only if the Frontend process is started.

  • Subscanend, Scanend  (always together with Cancel): Regular end of a measurement cycle



A graphical outline of the hard- and software connections one can find here !



Hardware Connections to the Control Units and Instruments

The Status- and Input-bits are connected to the different control units setting the right receiver and ULO configuration. They are handled by the „SPS“ and distributed to the VME processors via ProfiBus. The synthesizers and their frequency counters are connected via the IEC-Bus, in the same way as the frontend control units. The IEC-Bus is split by an IEC-Bus-Switch, which switches between two single busses, one for the synthesizers and the counters and the other one for the frontend control units (1-7).




The central rack in the control room of the telescope, where all connections to different units are hardwired. Here specially is shown the part concerning the connections for the frontend.




The ULO-rack with the control units, synthesizers and the frequency counters






The VLBA rack with the control units for VLBA and the receiver racks






One of the receiver racks with the frontend control unit. The ESM-Drive unit is controlled by the ESM control unit showing by the controllamp whether it is active or not.



The Console of the PowerPC Processor

In order to change the nonvolatile RAM of the processor, i.e. the boot parameters like the IP address of the processor and the server or the startup filename, one must have connected an alphanumeric terminal to the COM port 1 of the processor (at the backside of the VME crate). Changes of these parameters should be left to experts.

But there is another possibility to get access to the systemshell of the VME-Processor. Beside the connection of a terminal at COM port 1 one can also remote login into the processor from another computer. But only one user can have access to the system shell. If using remote login a user name with a password is obligatory. All in-/output from the processor is done by this shell and it can be interactively used to get information from the system or to start and stop programs (see here !).
The PowerPC processor has the name vme13 and its prompt in the system shell is frontend>.


Loading and Starting the PowerPC Processor

 
Hardware addresses used for certain modules or instruments are coded in a general initialization file, which is read in  and interpreted by the programs at startup. The location and name on the host computer is:

/home0/VMEload/Effbg_VMEsystem.config .

After Power On, Reset or (Re)boot the processor establishes a connection to the host computer. Then the VxWorks operation system module is loaded from the host and started.  IP-addresses, startup-filenames  and other parameters are taken from the nonvolatile RAM of the processor. Hereafter the system executes a script-file, which loads all necessary modules for the frontend processes, which will be initialized and started.

One of the first actions of the processor after loading is to reset the Signal and Time Generator, to read the actual time and set the system clock to that time, day and year. The VME module delivers the complete date (i.e. time,day and year). As IRIG B only delivers time and day of the year, the year is read from the init-file mentioned above. For comparison the program also reads the time of the host to get its year number. If the year of the host is greater than the year in the init-file, the host year is assumed to be the true year number. This procedure was chosen to avoid problems, if the processor is started early in the year and the file is not yet updated for the new year number, because also the time information of the host may not be exact.

For that reason it is advisable to check the year on the display of that module, especially if started early in January.

The frontend programs are automatically started by a script. Here follows a short description what is done in this startup script 

(/home0/VMEload/PPC3604/vme13startup):

  • First setting global variables and network definitions and establishing a RAM-disk for message-files

  • Loading  the object codes, the frontend structure and global variables into the computer memory

  • Initializing the time module and setting time of computer: (that can take up to 90 s, until synchronisation is found by the hardware)

  •   Starting the message handler, the ProfiBus, the sysclock handler and frontend data broadcasting

  • Initializing the Frontend

  • Starting the frontend control programs

 
The following tasks are then running in the VME processor:
(One can see that  by typing the command  "  i "  (for information)  or "taskShow(0,2)")

  NAME ENTRY TID PRI STATUS PC SP ERRNO DELAY

---------- ------------ -------- --- ---------- -------- -------- ------- -----

tExcTask excTask 37faa00 0 PEND 2494d4 37fa8e0 38001d 0

tLogTask logTask 37f7e60 0 PEND 2494d4 37f7d50 0 0

tShell shell 36da590 1 READY 242a88 36da150 30065 0

tWdbTask wdbTask 376aa40 3 PEND 23d140 376a910 0 0

tScsiTask scsiMgr 37e42f0 5 PEND 23d140 37e41e0 0 0

tNetTask netTask 37a64a0 50 PEND 23d140 37a63b0 0 0

tPortmapd portmapd 3775780 54 PEND 23d140 3775520 3d0002 0

tTelnetd telnetd 3772c00 55 PEND 23d140 3772a90 38001d 0

tTelnetOut_telnetOutTas 3ffd910 55 READY 23d140 3ffd5f0 0 0

tTelnetIn_3telnetInTask 3767be0 55 READY 23cf8c 3767860 0 0

FECANCEL establish_fr 36f3290 60 READY 2423ac 36f3150 38001d 0

FEP fep 36ee250 61 READY 2423ac 36eded0 3d0002 0

FEORG feorga 36e43f0 62 READY 2423ac 36e42b0 0 0

FESOCK fesocket 375a090 64 PEND 23d140 3759f20 3d0002 0

MESSG_HDL msg_start 3707f00 90 READY 2423ac 3707cc0 38001d 0

FRONT_HDL Run_Send_Fro 36fd0f0 91 DELAY 2423ac 36fcae0 0 361

CLOCK_HDL clk_start 3738c20 92 DELAY 2423ac 3738900 38001d 1177


  • FECANCEL: Initializing frontend and monitoring the CANCEL bit of  the control computer

  • FESOCK:  Communication with the control computer getting frontend parameters and sending messages and errors

  • FEP:  General synchronisation and administration of the process phases commanded by the control computer

  • FEORG:  Setting and communication with the connected instruments

  • FRONT_HDL:  Delivering frontend information via broadcast to the local net

  • CLOCK_HDL: Handler to control the system clock


The FEORG task decodes the content of the system init file (copied to memory) and searches the VME modules and the connected instruments on the IEC-Bus. At the same time the FESOCK task establishes a socket connection to the control computer, if the server program is running in the control computer.
The FRONT_HDL tasks establishes a server process at a predefined port regularly sending the actual frontend parameters via broadcast .

The CLOCK_HDL process is a handler, which regularly reads the time delivered by a hardware time module and compares it with the computer time in order to reset this time if necessary.



If the startup of the PowerPC has finished, it prints out messages about the configuration of the frontend.

At that point one can start or restart the frontend program in the control computer, which restarts some tasks in the PowerPC to initialize the instruments. Testing all IEC-Bus instruments (i.e. Synthesizers, Frequency counters and Frontend Control Units) takes a while. Progress report is send to the control computer. That can take ~3-4 minutes.

If the PowerPC processor is already running and has finished its startup sequence, one can directly start the frontend program in the control computer.



Communication between Control Computer and PPC Processor

The FESOCK task connects as client to a compagnon socket server task in the control computer, which has to be started before FESOCK. The server receives commands via the socket connection and sends back the commanded string or an acknowledge message. Possible commands are: To send the parameter field of the frontend, to tell that a control phase has finished or that an error occured. The frontend control program in the control computer sets single bits in an output register marking the ongoing control phases ( init, setup etc. ). This output register is connected to an input register in the VME crate, which is continuously monitored by the FEP task. The completion of a phase is signalled to the control computer by sending the adopted return. The FECANCEL task loops on the CANCEL bit set by the control computer.
 
 

TEST Programs in the PPC Processor

The following programs can be used to test the function of single instruments. They are available, if all frontend programs have been loaded. The programs can interactively be called under the system shell of the PowerPC processor with the following commands:

  • switch4842_test_port(port_nr) [port_nr = Portnumber of switch (1...3)]

    Looking for the IEC-Bus-switch and setting the typed portnumber.

  • EMSE_test(unit)      [unit = number of frontend control unit (1....7) ]

    There will be a continuous communication between the processor and the frontend control unit.

  • RACAL_test     

    Tests all synthesizers defined in the startup system file.

  • HP5386_test       

    Tests all HP frequency counters defined in the startup system file.

  • Show_Frontend_Initxt

    Printouts the contents of the system init-file, which is stored in memory

  • Startup_Errors

    Printouts the messages of the last configuration phase

  • reflMem_init

    Tests reflective memory module.

  • tip600_test

    Tests the TIP600 digital input board

  • reset_sysclock

    Tests for the IRIG-B time-module, and resets the computer time to that value

  • stop_sysclock_hdl, start_sysclock_hdl

    Stopping or Starting the CLOCK_HDL task.

  • Start_Send_Frontend(debug) [ debug != 0, printout of debug information]

  • Stop_Send_Frontend

    Starting or Stopping the FRONT_HDL task

  • Run_Send_Frontend(debug) [ debug != 0, printout of debug information]

    Run the frontend broadcasting interactively in the system shell.

  • Restart_Frontend, Stop_Frontend, Start_Frontend

    All frontend tasks without FECANCEL will be killed and again restarted with the normal startup procedure. Before a (Re)Start_Frontend the FECANCEL task must already run.

  • Remove_Frontend

    The FECANCEL task is finished and the memory for the text of the system init-file is freed. That must only be done after a Stop_Frontend

  • Init_Frontend

    Restart of the FECANCEL task with a new initialization of all frontend global parameters.


Setting and resetting more printouts in system shell for debugging:

  • (re)set_feinit_debug

  • (re)set_fep_debug

  • (re)set_feorga_debug

  • (re)set_fesocket_debug



A description of the software in the PowerPC processor one can find here !


ERROR Messages

Messages at startup concerning serious VAX system errors:

In such cases a restart oft the frontend processes may help !  >>>  @VME_FRONTEND or FEP
Otherwise try again the SYSTART procedure.

VMEFRONT-FATAL ERROR: UNABLE TO INSERT VME-FLAGS
FRONTEND-FATAL ERROR: UNABLE TO MAP BACKEND AREA
FRONTEND-FATAL ERROR: UNABLE TO MAP ACTIVE FIELD
FRONTEND-FATAL ERROR: UNABLE TO MAP PASSIVE FIELD
 

Messages concerning problems with the VME processor or connected hardware:

These messages concern a serious problem with the Frontend configuration file (see above), which is not available or corrupted:
FE-Config-File NOT FOUND
TEXT or PARAMETERS in FE-Config-File WRONG

In the following cases also a restart of the frontend processes may help !  >>> @VME_FRONTEND or FEP
If that does not solve the problem, first try to restart the VME processor and wait until it displays the message to restart the VAX frontend process. Then start that in the VAX:  @VME_FRONTEND

VME-PPC: SERIOUS PROBLEM WITH VME-PPC CONFIGURATION
VME-PPC: PROBLEM WITH CONFIG FCUS AND ULOS
Problem STARTING VME-PPC Frontend-process !
Problem STARTING feorga-process !


If these messages come continously,  some instrument can be hung up. First restart the VME processor and wait until it displays the message to restart the VAX frontend process. Then start that in the VAX:  @VME_FRONTEND or FEP
VME-PPC: PROBLEM WITH INIT
VME-PPC: PROBLEM WITH SETUP
VME-PPC: PROBLEM WITH HALT
TIME OVERFLOW in executing Phase ...
 

Messages concerning problems with the VAX hardware (CAMAC):

The CAMAC output register 3074, which delivers the synchronisation bits to the VME processor may not be available or has problems.
PROBLEM WITH SYNCBITS OUTPUT MODULE!
 
 

Messages concerning problems with the ULO and the frontend control units:

The unit number for the frontend control unit is wrong:
Frontend-Control-Unit ... outside possible range

Serious problems with the frontend control units:
Problem in Communication with FCU ...
FCU unit ...  NOT AVAILABLE
Problems setting Frontend-Control-Unit ...
FCU UNITS NOT FOUND: ...
FCU unit ... NOT FOUND

The MANUAL/REMOTE switch is set to MANUAL:
ERROR Frontend-Control-Unit ..., SWITCH NOT REMOTE

Serious system error, the switches of the ip488 module are set in the wrong way:
ip488-Device IS NOT Controler, addr: ...

The named frequency counter (with address) have not been found by the program:
NOT FOUND COUNTER: ....

The named synthesizer (with address) have not been found by the program:
NOT FOUND SYNTH.: ...
 

Other messages:

During the MEASURE phase the VME processor reads the frequency counters to compare the calculated frequencies with the real ones. This message can be a hint to a serious problem with the ULO frequencies or only a problem in reading the frequency counters:
NO SUCCESS IN READING FREQUENCIES

 Only telling you that the VME processor is now setting up the frontend system:
... ATTENTION: GIVING CONTROL TO VME-PPC ...

The VAX demanded a new configuration cycle of the VME processor:
VME-PPC: FRONTEND - PROGRAM KILLED

You have commanded to use the Frontend control unit not under computer control:
Frontend-Control-Unit ... used in MANUAL MODE
 

Messages telling the state of the VME processor:

VME-PPC NOW RESTARTING FEORGA PROGRAM
VME-PPC IS NOW CONFIGURING SYSTEM
feorga: process will RESTART !
.. NOW CHECKING ULO FREQUENCIES ...
... SETTING SYNTHESIZERS ...
... SETTING FRONTEND CONTROL  NR.  ...
LOOKING FOR SYNTHESIZERS,COUNTERS AND FCUS ...
ALL SYNTHESIZERS AND COUNTERS FOUND
====>> AVAILABLE FCU units: ...
 




jneidhoefer_at_mpifr-bonn.mpg.de