Troubleshooting Tips

Possible Setup issues

Hub

PRI/T1 card does not initialize. No GREEN indication on screen (“D Channel Established” message or “RBS Link is Up” message not displayed.)

    • Protocols don’t match.  Contact the Telco provider to verify settings
    • Telco did not turn up D Channel on PRI.  Contact Telco provider.
    • Check CSU unit.  Make sure there are no alarms or LOS.  If so track down causes.
  • Check the NetAccess card. The green led should be on (Board running), the FAIL led (board failure) should be off and the amber light, which represents the loss of signal, should be off.

Loss of signal from the Telco

Some Telco’s are using an automatic switch-off system if they detect a loss of signal from the 2101 side after a few minutes (15 in general). In the event they lose the signal from the 2101 they will shut down their end as well.  That is why it is important to check the CSU and figure out if there is a LOS and on which side the loss is being reported from.

Studio I/O/Extended Hybrid

Audio issues => Audio menu

  • No input source: check the INPUT SOURCE (Analog/Digital). If Digital, check the AES SYNC parameter also.
  • Callers can hear themselves: check that the ROUTING setting is where it is supposed to be.  Most issues of callers hearing themselves is due to an improper Mix Minus feed from the console.
  • Callers on different Hybrids can hear themselves.  Check the Routing to determine if this is what the customer expects.  If the hybrids are supposed to be SEPARATE and they are NOT supposed hear each other then suspect a Mix Minus problem at the console.  Else the Routing option is set for MIX.
  • DJ feedback/Poor Hybrid null/audio “hollow” or “tinny”: check that FeedThru options are OFF.  (This option may no longer exist in later revs of sw)  If they are OFF, check the STUDIO IN level on the front panel meters. If they are higher than –8 dB (1 LED equals approx. 2 dB), the Hybrids might not work correctly. Thus, the input signal gain should be decreased. Check the Input level and set it up correctly according to the customer equipment.  If the input level is set to +8 and the peaks are still hitting 0dbfs you must use a pad on the inputs to reduce the gain.  Try to keep the setting at +4 max.

Note:  The Front Panel meters can also provide valuable information.  The left meter shows the caller audio and the right meter shows the Send to Caller audio.  The small black button within the windows toggles the meter function to show audio before and after the hybrid processing.

Telco issues => Telco menu

  • Audio from caller distorted: check the Telco settings on Hybrid.  ETS300 should be used for Europe (A-LAW) and all the others (AT&T or Nat. I-1) for North America (MU-LAW).

System issues => System menu

Here is what to check if the Studio I/O is in a ‘Waiting For Hub’ state (seen on the DDs):

  1. Check that the network cable is plugged in.
  2. Check that there is no conflict with the IP address on the 2101 network.
  3. Check that the HOSTNAME is unique and does not have any SPACES in its spelling in the 2101 Configuration Utility.  (special characters are not allowed ie. &$#@ etc.)

NOTE: when using the serial port for log purposes, check that the Serial bit rate is correctly set up on the Studio I/O.

Desktop Director

Often, the customer is confused by the DIRECTOR mode (Producer/Talent).  Director mode can answer calls to handset only.  Talent mode always controls the hybrids.

Possible issues when updating Software

Hub

Here is a list to follow in case of a failure during the Hub update process:

  • Check that the user computer can ping the Hub that has to be updated. If it cannot, the customer has to set up his workstation and/or network correctly.  Verify the computer settings from a command line by typing “ipconfig”.  It will report back the computer’s set IP addresses on the installed network interfaces.
  • When using the OLD 2101 Configuration Utility the user’s computer OS must be NT based for the utility to work properly.
  • Check that the user computer is running the FTP Server program or updater package.  If the following message appears within the FTP program ‘ERROR: Unable to open port 21…’ when attempting to launch the FTP Server Program it means that another FTP Service is already running on the PC.  Only one service at a time can access port 21.  If the user computer is running an FTP server in the background you will need to disable it.  Go to the Control Panel and find the Services icon. Verify that Windows is not running an FTP service of its own.
  • Check the root path of FTP server on the PC is pointed to the proper nested directory structure.
  • Check the FTP site settings in the Web Page. It has to match with the user computer IP address (Software update page).
  • Check the FTP path in the Web Page. It has to be the following: ‘Pub\2101\Updates\winsim.exe ‘
  • When you are NOT using the Current software Updater FTP package and doing a manual FTP update check that the ‘winsim.exe’ file can be found at the following path (user’s PC): ..<root directory>\pub\2101\updates\.  If you are using the embedded 2101 updater package you cannot check this.  By default it should be encoded with the correct path.
  • If the update went okay but you see no difference in the old version to the new check if the customer rebooted the Hub after the update process to the correct software bank (REBOOT key in the Software Update page).

Studio I/O and Extended Hybrid

Here is a list to follow in case of a failure during the update process:

  • Check that the user computer can ping the Hub that has to be updated. If it cannot, the customer has to set up his workstation and/or network correctly.  Verify the computer settings from a command line by typing “ipconfig”.  It will report back the computer’s set IP addresses on the installed network interfaces.
  • When using the OLD 2101 Configuration Utility the user’s computer OS must be NT based for the utility to work properly.
  • Check that the user computer is running the FTP Server program or updater package.  If the following message appears within the FTP program ‘ERROR: Unable to open port 21…’ when attempting to launch the FTP Server Program it means that another FTP Service is already running on the PC.  Only one service at a time can access port 21.  If the user computer is running an FTP server in the background you will need to disable it.  Go to the Control Panel and find the Services icon. Verify that Windows is not running an FTP service of its own.
  • Check the FTP site settings on the Studio unit. It has to match with the user computer IP address (SYSTEM menu).
  • Check that the Studio unit is running on the primary bank (SYSTEM menu). If it is not, select Boot Bank as primary and reboot the Studio unit.
  • Check the root path of FTP server on the PC is pointed to the proper nested directory structure.  
  • When you are NOT using the Current software Updater FTP package and doing a manual FTP update check on the user’s PC that the ‘image’ and ‘hybrid’ files are in the following directory: ..<root directory>\pub\2101\updates\<files here>
  • Check if the customer rebooted the Studio I/O after the update
  • Check that the path in the Studio I/O is set to “\pub\2101\updates\image.”  You will actually see this path come up in the FTP window when the units attempt to find their software.  If it is incorrect the Studio I/O will not update.  You will need to Telnet to the Studio I/O and change the path using the FTPpath command.  See your manual for full details on the telnet commands.

Director (EPROM change)

  • Did you insert the EPROM correctly?  Goes in only one way.  If you forced it you probably damaged the socket.

Various Problems/Symptoms of Failure

Hub

Cannot communicate with the Hub on the Network.

  • Check the network port on the CPU card.  Is the Ethernet plugged into the correct port (bottom for the Rocky MB’s)?   Any LED’s lit?  Could be bad port on CPU card, bad cable, or bad port on Ethernet switch.  
  • Ping the Hub with your workstation.  Does it respond?  Is the IP address entered for the Hub correct?
  • Ping the I/O’s from your workstation.  Do they respond?  If not maybe your PC has a TCP/IP config problem (wrong IP address range, wrong subnet, you configured the wrong network device).   Verify the workstation’s TCP/IP settings  (type “ipconfig” from a command line).
  • Try bypassing the Ethernet switch.  Connect a crossover cable directly from the 2101 Hub to your workstation.  Verify the IP address on the workstation is in the same range as your hub is supposed to be.  Do the ports show activity on the LED’s now? Can you ping the hub?  

Hub shows activity on the network but cannot connect to Web Page.

  • Can you telnet to Hub?  Standard telnet session will bring you to the root directory of the hub.  Login should be Administrator/Administrator.  If you cannot get a Telnet session then the OS may not be up and running.
  • Can you telnet to port 123?  This is a maintenance port.  Login is “sesame”.  If you cannot you need to determine if the OS is truly booted up and running.

To verify that the Hub booted up to the NT OS plug any monitor directly to the CPU card of the hub.  If the NT shell booted up you should see a BLUE screen with white text across the top.  This should indicate that the hub booted to NT but shows NO indication if the 2101 application is running.

If you can PING the hub then you need to verify that the 2101 application and web browser apps are running.  Use Netmeeting to connect to the 2101 Hub remotely (see part 5 of this document).  If you can connect with Netmeeting you should see 2 applications running on the remote desktop, one is the 2101 application and the other should be the web server.  The web server should be a DOS looking window.  Try manually closing the web server application by clicking on the upper right X.  The 2101 application should automatically restart the web server app.  Try connecting again.  If you still cannot connect to the web page try specifying Port 80 on the URL address line in the Web Browser (ie. http://<ipaddressofhub>:80).  If you still cannot connect then there is something deeper within the network preventing your access.  Refer to the developers for assistance.

You can also attempt to connect to the Hub via the OLD Config Utility software to see if you can talk to the hub.  This may not indicate that the 2101 application is running but it may gain you access to the hub.  You must first authenticate yourself to the Hub.  Make sure to connect to the Hub by a Search command from Network Places. Then, if found on the network, enter the following login and password:

  • Login: Administrator
  • Password: Administrator

PRI/T1 card has some LED’s lit but doesn’t initialize to Line.

  • Check the NetAccess card LED’s. If the amber LED is on, this means that no signal is being received from the Telco.  Check the PRI/T1 line from the Telco at the external CSU unit.  See if there are any errors displayed on the CSU like LOS (Loss Of Signal).
  • Protocol not configured correctly.  Card can’t properly initialize if the protocol is wrong.
  • Length of cables: check that the T1 cables have the right length.  There is a termination setting within the PRI/T1 configuration for proper termination.
  • Is the PRI/T1 turned up at the Telco provider?  Telco provider may have the D channel turned down.  

PRI/T1 card doesn’t appear to be working.  No LED’s lit.

  • The 2101 application is not properly initializing the card.  Select the REBOOT option in the web interface if you can access it.  If the card does not appear to initialize it could have crashed. The only way to reset it is by a HARDWARE reboot.  Press the RESET button on the front panel of the Hub or power cycle the Hub.
  • If the card is still not coming up, power down the hub.  Reseat the PRI card in the back plane.  Try again.
  • If the card is still not coming up, it could be a problem with the PRI/T1 card, OS, or 2101 Application.  Connect a Monitor to the CPU card and verify that the NT embedded OS is booting up completely.  If it properly booted up you should see a BLUE screen with some white text lines at the top.  This is normal for the Hub.  If the monitor to the CPU card shows the Hub constantly rebooting to the post screen or is frozen on the post screen then either the BIOS is not set correctly on the CPU card or the FLASH Hard Drive is corrupted.  Try reseating the FLASH card on the CPU card.  If this doesn’t fix the problem get a new flash card with a clean 2101 image burned to card.  Verify the BIOS settings are correct for the MB.  (1st Boot device should be HDD-0.  All other boot devices disabled.  All other BIOS settings are factory defaults)
  • If the OS appears to be up and running and the Telco card is still not initializing connect with a Netmeeting session.  Check to see if the 2101 application is running.  If it is not possible the Hub Spawn application may have crashed.  This application’s function is to restart the 2101 application automatically should it stop either manually or if the main application crashes.  Look at the Task Manager of the hub to see if HubSpawn is loaded properly.
  • In the Netmeeting session you see the 2101 application opening and closing.  This is possibly a corrupted application or registry settings.  Consult with an engineer about how to fix this.  You may be able to navigate thru Windows to stop HubSpawn then move some key files around and see if you can get the unit functional again.  Navigate to the Registry to check if any corruption happened to these settings.  May not be repairable in the field then it would require a new flash card altogether.

Quad-Link Cards have nothing plugged into them but no LED’s lit.

  • This is a possible indication that the 2101 application or the NT OS has not started properly.  Connect a monitor to the CPU card and verify that the NT OS started correctly. (see procedures above for testing)
  • If the NT OS started correctly, try reseating the Quad Link cards.
  • If the card(s) are still not coming up, restart the hub via the front panel Reset button.
  • Verify that the 2101 application is running.  Use Netmeeting to the connect to the 2101 Hub remotely.  (See part 5 of this document for details)

Note: There are two different types of Quad-Link cards being used in the field: revision A and revision B.  Revision A requires a custom “Crossover cable” to connect to the Studio interface. See the manual to differentiate Rev A and Rev B. Earlier customers have the Rev A version.  There are at least 2 systems out there like this.   Most everyone else has the rev B style.

Quad Link Cards have cables plugged in, Ports show LED still lit.

  • Possible bad Cable, Bad Port on Quad Link card, Bad T-Link Card, or I/O is not ON or plugged in.

Outgoing calls not always getting routed.

  • The symptom experienced is that outgoing calls are not working all the time and right after they failed, a line is ringing.  If you log into the 2101 Hub via a telnet session and monitor inbound calls (as noted in Part 5) you can verify the B Channel assignments on the incoming calls.  

Incoming calls on the PRI/T1 are typically processed from the Bottom-Up (starting on B Channel 1, though we have seen this opposite in Europe).  In case of a collision (Glare) when an inbound call is trying to come in on the same channel an outbound call is being made, according to the Q.931 protocol (PRI protocol), a priority is given to incoming calls.  Depending on the Telco provider, if there is a conflict of inbound and outbound call ordering the Telco provider may be able to change their inbound call ordering.

OR you can correct this glare issue on the 2101 hub in the System Options of the Web Browser.  By default we are set up for Top-Down ordering for outbound calls.  (This option modifies the registry value in the hub OutgoingScanOrder in the key Software/TelosSystems/2101.  If it is set to 0, it means that the outgoing calls are done from the Top-Down. If set to 1, the calls are done from Bottom-Up).

On some occasions the Telco provider has sent inbound calls in a CIRCULAR order.  If you verify this type of ordering get the Telco to change it to Bottom-Up hunting.  

Note: The Hub application has to be restarted for the change to take affect.  Go to the Software Update page to restart the application by pressing the REBOOT button.

Studio I/O/Extended Hybrid

LCD Screen on Studio I/O is flashing

    • It may mean this is a Power supply issue. It could come from a faulty P/S or from too many DD’s connected to the unit without any external power supply.  Disconnect all DD’s and see if the problem stops.
    • Unit constantly rebooting.  Disconnect the T-Link cable to Hub.  If unit stops rebooting then there is a possible bad T-link card in Studio I/O, bad port on the Quad link card in the Hub, or the cable is bad.  If it doesn’t stop rebooting, memory is possibly corrupted.  Try a new SIMM module.

No caller Audio from Hub on any studio I/O

  • Suspect Ribbon cable between PRI/T1 Card and Quad Link cards.  
  • Suspect the PRI/T1 Card itself.
  • If there is no audio on I/O’s sharing the same Quad Link card and other I/O’s are okay, suspect Ribbon Cable or that Quad Link Card.
  • Suspect the wrong protocol setup on the PRI card.
  • Check the Studio ISDN setting  (Mu-Law or A-Law setting)

Callers to 2101 hear other audio rather than ringing or NO ringing sound when they call to the studios.  The 2101 receives calls ok.

  • For the PRI the 2101 is supposed to generate the ringing or busy sounds to the caller if the calls reach the 2101 system. The Hub itself does not create the Ringing sounds but assigns a studio I/O as the master RING SOURCE for the system.  A rare bug in the I/O code will cause the ringing audio to redirect itself sometimes to the MOH, input audio or no audio of that particular unit.  The fix is to reboot the I/O’s one at a time till the Hub sees the primary Ring Source go away then reassigns the Ring Source to a different I/O.  Rebooting the Hub is typically not necessary.

See calls ringing in on DD but cannot answer them on DD or to Hybrid.

When this occurs typically means that there is NO audio path to the hybrid.  A bad T-Link card, Quad Link card or bad cable will exhibit this symptom.

  • Check the LED’s on the T-Link card and on the Hub’s Quad Link card. Its corresponding LED should be turned off. If it is ON or blinking erratically, the T-Link card might be faulty or the cable between the Studio unit and the Hub might be faulty, or the port on the Hub’s Quad Link card may be bad.  No path for the audio will prevent you from answering any call.

Poor/Distorted Audio on Hybrids and Handsets.

  • Check T-Link/Quad Link cabling for bad cables.  If possible move Studio I/O as close to hub and shorten T-Link cable run to 1 meter and try again.
  • Check T-Link/Quad Link cards for a bad or intermittent port/card.
  • Possible bad Ribbon Cable in 2101 Hub.  Reseat or replace.
  • Possible wrong setting of audio LAW Mode on Hub or Hybrid. Check Hybrid Telco Settings.  Check Hub’s setting.
  • Possible problem with audio quality coming from Telco provider (rare but it has happened).  

Directors (Desk, Console, Call Controller)

Desktop Director says “WAITING FOR HUB”

    • Check the network port on the Studio I/O. Is there LED activity on the port? If it is off unit is not connected to 2101 LAN.  Check cables, switch.
    • Check that the Studio unit is not locked up.  Press any MENU key (below the LCD front panel). If nothing happens, the unit is frozen. The only way to reset it is by performing a hardware reboot.

Poor Audio on DD Handset.

  • Bad audio on handset.  Swap the DD with another one to make sure it is coming from the DD instead of the Studio unit.

LCD on DD shows only Dots

  • Lines of dots on the LCD front panel means that the DD is seeing the Studio unit but cannot finalize the connection properly.  Unplug/plug back the DD. If the connection still does not come back, swap the DD with another one. If this doesn’t resolve the problem suspect a bad port on the DD card in the Studio unit.

DD’s LCD stuck on “TELOS DESKTOP DIRECTOR”

    • This means the DD rebooted correctly but it does not see any connection with the Studio I/O.

DD Frozen, Cannot navigate menus

    • DD frozen: it rarely happens. Most of time, this is related to the Studio I/O. Unplug/plug back the DD. If it comes back up properly, watch the Studio unit for signs of instability.  Note that there is a delay after rebooting the DD until you can begin to access the menu choices.  The DD will be communicating with the Studio I/O and establishing a connection.

Phone lines not showing up on the DDs

    • Verify that the show is actually loaded to the I/O by selecting the show from the DD.  The LCD of the DD should display the I/O host name and the show selected along the top edge of the display.
    • Confirm that the programmed numbers entered in the assigned Show are matched to a programmed DID/phone number in theTrunk config of the PRI/T1 card.  Phone numbers not previously programmed to the Trunk Config will not work in any programmed show.
    • Phone numbers have channels assignments in the Trunk config and line appearances of the programmed numbers in show exceeds the number of channels associated with itself. (ie.  If 1001 is assigned to channels 12-15 you can’t have more than 4 appearances of 1001 in a show).

Lines appear on DD’s but calls into the 2101 ring busy

There have been times when the Telco states that they are sending X number of digits in the Call Setup but in actuality they lied to you.  There is a “trick” to avoid calling the Telco (which might be a big loss of time, knowing how busy they are):

  • From a workstation on the 2101 network, Telnet into the Hub using port 123 (Telnet <Hub IP> 123)
  • Enter the following login: sesame
  • Type log 82 (This is the inbound messages received by the Netaccess Board from the Telco Provider)
  • Perform an incoming call to the 2101 (with one of the phone numbers given by the Telco)
  • Look at the inbound call data and find the parameter for CALLED PARTY NUMBER.  It will display the number of digits used.  See if this matches what you entered in your Trunk config.  If it doesn’t, modify the Trunk config and RESTART the 2101 application to apply the changes.

See calls ringing in on DD but cannot answer them on DD or to Hybrid.

When this occurs typically means that there is NO audio path to the hybrid.  A bad T-Link card, Quad Link card or bad cable will exhibit this symptom.

  • Check the LED’s from the T-Link card. Its LED should be turned off. If it is ON or blinking erratically, the T-Link card might be faulty or the cable between the Studio unit and the Hub might be faulty, or the port on the Hub’s Quad Link card may be bad.  No path for the audio will prevent you from answering any call

‘No Show assigned to Studio’ message on the DDs.

    • From the DD check to see if the assigned shows to the Studio I/O appear as options to be selected from the DD’s menu option.  If they are not verify that the Studio I/O hostname is correctly entered into the Hub’s configuration and that the studio has shows assigned to it.
    • If you see shows but can’t select them make sure there are no spaces or special characters in the show name.  Spaces in the names will prevent shows from being selected to the I/O

Audio Dropouts or Clicks

  • If you have multiple PRI’s this could be a clocking problem.  Telco/PBX must be the clock master and sync’d to the network, not the 2101.
  • Possible T-link/Quad Link card problem.  
  • For Extended Hybrid possible bad S Interface card or bad DD card in main interface.

Assistant Producer

The customer has to be familiar with the Connection menu. A lot of times, when AP is in a ‘Not connected to a show’ state, the user gets confused.  They will be connected to the Hub but have not selected a Studio I/O.   When logging in the first time they connect to the Hub but then they need to select the Studio they wish to screen.  This is NOT automatic the first time around.

Another issue is the number of AP connections, which is set up to a maximum of 1 per DD and unlimited stand alone connections.  Be aware that the more connections there are the more network traffic to deal with.  Keep the number of stand alone AP connections per studio to a minimum.

Connection Lost message.  System dropped all calls suddenly. Dots on DD’s went away then came back.  Happened when connecting with Assistant Producer for first time.

This is a bug with the older 2101 configuration utilities.  The original config utility and the first generation of web browser caused these problems when the users were modifying and deleting Studio I/O.  

The cause is a corrupted registry entry in the SHOW HOSTS section of the 2101 key in the 2101 Hub.  Because of how the AP Client connects it is supposed to poll the Hub’s registry for studio I/O’s and their listed available shows.  If the list is misnumbered or there are orphaned entries in the registry and it no longer matches to the HOST SHOWS key the AP server in the 2101 Hub crashes and the 2101 application crashes along with it.  HubSpawn will restart the 2101 application but the result is a loss of all current calls on all Studio I/O’s.  Rather annoying.

This problem has been corrected on the newest Web Browser as of this date.  Upon running the web browser the browser is scripted to check the registry keys to make sure they are correct.  If it finds a mistake then it will force the user to correct the problems automatically then continue on into the normal configuration web pages.  

There are steps to manually correct this problem if the customer does NOT have the current web browser.  Either by saving a text file of all the settings thru the OLD config utility then modifying the text file manually then reloading the file, or manually editing the 2101 Hub’s registry can repair the problem.  Care needs to be taken for each of these procedures.  Contact the developers for assistance before proceeding with this type of repair.  

Note: this problem only manifests itself when the AP client is used.  Any other time the 2101 Hub is fine.