Phones @ “Work”: Donald Trump’s (Oval) Office

Thank you Joe, the UCx guy for reminding me to do an update with Trump.

If you were on the ether hashtag campaign of “#MakeAmericaGreatAgain” or “#MakeAvayaRedGreatAgain” or “#MakeAvayaGreatAgain”, don’t be hopeful on the latter two hashtags after recent events from our new President and the incumbent phone vendor in the White House.

Continue reading

POTD: Local Papa Gino’s

Sadly, where I live, Avaya or Nortel isn’t “alive and well” unlike another site I follow. Nortel has disappeared in my state in public and private entities in lieu of Cisco years ago and Avaya Red has slowly disappeared too.

On a Christmas Eve tradition before I was born, my family would order pizza out at the local Papa Ginos, that is local chain with more than one hundred stores around the Greater Boston region, basically in four of the six New England states. It’s reputation is fresh quality pizza of with quality ingredients. Over the years Papa’s has had exclusive marketing deals with the local Boston teams such as the Red Sox and currently the Patriots.

The chain has used AT&T products going back to the days of Western Electric. This location I had frequented growing up had used one of those 10 line 1A2 wall mount Key telephones till a cutover around 2001 to a Partner ACS system. The only ComKey I’ve ever seen in production was another store nearby, and that had cutover to a Partner circa 2001 or 02.

I’ve been to mostly the New Hampshire stores, and D’Angelo the sub shop, is a sister brand to Papa Ginos. I don’t recall them using any phone systems, the one nearby me, that I took a few years back with an Avaya van uses POTS phones.

But today, just the next block away from that same D’Angelo, I noticed  this phone. Nope, its not a 9600 Avaya IP or 9500 DCP set. No, worse a Polycom VVX 310 set. (I haven’t been here for a while, some days I normally walk here because it’s not that far away from my home.)

img_1831

Continue reading

SIP Experiences, part two

SIP is like a PC, it’s a great technology. It’s like “I wanna have a computer on my desk but I dunno what I really want out of it”. As the 1990s came along, these little things became a nightmare for network administrators. Not only that but PCs had too much power for what many people didn’t need. What I am talking about is enterprises not consumers.

SIP is in fact much like a PC.

Continue reading

Rants: Consumer’s and the Lack of Understanding Telephony

 “I also want to express my appreciation at being able to share my telephony passion with someone who actually understands what I’m talking about.  I hear all the time, people want to be able to walk up to a telephone, pick it up, dial and it just works.  They don’t care about the why or how it works.  They don’t wax eloquent about the beauty of a route pattern, or think long and hard about survivability, or stop to ponder why things do what they do, and what historical precedent may be contributing to the current behavior.” Recent private email with Jason

Today’s rants is all about the “consumer”, those pesky “end users” and how those same “consumers” fail to understand IT, IS or telephony and blame the boogy man of cable companies for not responding to calls on a very unusual series of events.

If you were one of the lucky victims of loosing Comcast Cable TV today (markets impacted were Sacramento, San Francisco, Houston, Denver, Detroit, Boston/Manchester, and other markets), you’re not alone. Supposedly Comcast had a breakdown of transmission of several cable networks today. It was resolved within a couple of hours on my Comcast (formerly Adelphia) headends by noon.  Unconfirmed reports claim Internet services were down in some regions. I didn’t notice today, I had my set off, until my grandmother was telling me she couldn’t get the Fox News Channel, and assumed maybe was an outage on the Newscorp stations. I then did a search to find out, oh yeah there was an outage. A big one.

Regardless my mother had tried to call Comcast and we got “All Circuits are busy” announcement followed by a PSTN-equivalent reorder tone. My mother never heard such thing in years, probably in the days when we still had SXS switching in the 603 area code. (ESS wasn’t fully available till 20 years ago!) I called my cell to see if the modem from 2010 was acting up again. I heard my iPhone go off.

The problem is people are moaning and b-tching about “I lost my cable and my call gets dropped” blah, blah blah.

The problem is that Comcast gets a load of calls to begin with. I bet nearly 70% of the Internet related issues is because the Internet is just a complicated protocol and because the router lives inside the home, per to Part 68 like rules, the ISP really doesn’t want to deal with keeping the modem/router/gateway reliable – it’s the customer’s responsibility.

Today was a problem in itself. The Cable TV services that typically doesn’t fail. For me, I was able to get my local stations plus the regional sports networks and ESPN, etc. (Most of my local stations broadcast through fiber optic lines to the cable providers. Believe it or not, the telecommunications standards of using fiber over dishes is becoming a standard fare.)

Because of this very unusual abnormality, the lines were loaded. I know we have a major call center out of Manchester for north of Boston customers, and I believe there are other regional call centers that answer calls to the 1-800-COMCAST. As I previously describe in the definition of the PBX, the forumla with ACD PBX systems is lets say there is 100 person call center, and you add 100 lines, then have about 50 to 70 more to have overflow or to “queue” calls, and enough lines to escalate to Tier 1 to 4 support centers. No call center in the world is designed to take every call in the world at the same time, because you don’t have enough people to answer and/or you would have to many people in the jail cell waiting to be answered.

Also noticing the NBC Universal side of the business going to Cisco (after being a loyal Avaya Red customer since the time of Divestiture), I believe Comcast/Xfinity have used Cisco even before they bought NBCU. Imagine the call center using Cisco Call Manager (the PBX/server version) and there is a bunch of overloaded calls coming into the CallManager. The system more than likely will crash depending on how much load the server can take at a time (roughly a CallManager can take about 4000 users among a series of servers.) If there was so many outages on the Cable TV side that flooded their call centers, I’d suspect the callmangers started to fire up and when these things crash and burn, it’s not a Definity where it can boot up within 5 to 10 minutes at a maximum. This could take up to 30 minutes plus the time to reboot the hardware (phones and gateways) and rebooting some Cisco routers or gateways takes a real long time. But again how often would this happen?

This is one of the deadly sins of Voice over IP. It is designed to be a fancy application running on a PC. PCs are a godsend if it works properly. PCs weren’t designed to handle telephony by design. Over the last decade it’s gratefully improved, but it is what it is. And depending on the hardware setup (SSDs vs magnetic hard drives, if RAID is used or if it’s a virtual machine is there enough resources) can be a factor.

Another thing to add is another law of telephony physics is called the Busy Call Completions per Hour or BCC or BCCH. For whatever reason in the digital age of telephony, calls can’t be made and placed at a single time. During small (but large to us) earthquakes, lines will jam up and what happens is callers will hear dead air. You can make DTMF tones, but that’s all you’ll get till the ESS switching gets restored. Is this necessary? I’m not a PBX or ESS engineer, but for whatever reason, most landlines can handle 100,000 completed calls per hour. (This is very important because what if  someone is calling their friend just to “check in” and you were the last completed call and another person is dying of a heart attack and they are the one that will hear empty DTMF tones to 9-1-1.) This is why it gets repeated during a natural disaster or an unusual event (like an earthquake on the East Coast) to not use the phone unless its to call someone like a 9-1-1. That’s also to say do not call 9-1-1 for non emergencies during major events. In the February 1993 bombing of the World Trade Center, before 3-1-1 was ever a common number, the New York City Fire and Police departments were indicated with calls, despite the New York Telephone network working without massive outages like someone not blowing the BCC limit, it was their 9-1-1 system that was blowing the BCC limit. (of course I am talking about the rest of the Manhattan island and the other four boroughs – it was given that lower Manhattan had little dial tone.)

In closing, it would not be unusual for an event like today to have calls drop and get announcements from the provider that “all circuits are busy.”

Another problem that people are not familiar is this “triple play” technology is no different than having SIP, over the top or broadband Internet. This is called DOCSIS, the protocol behind “triple play”. Basically all your services run on an IP network, but you don’t see it that way is because by design it works like a traditional cable box, or your phone works like an ordinary wireline service or your Internet is a high speed Ethernet connection.

The dirty little secret? All of this runs on an IP network. Not like your civilian/corporate network, but instead of having a Cat 5 cable to plug into a wall like Roku, it terminates via coax, and basically the coax goes to the “last mile” boxes in your neighborhood and through fiber goes to your headend. But it’s at the headend the “magic” of SIP like VOIP, and OTT like video is being processed. In short, your headend and your devices are basically the broadcast equivalent of the Internet of Things or IOT. I am not indicating that Comcast had a breach to their video network, but the chances of a hack are typically unlikely because a lot of this is done through secure networks and the least amount of hardware possible. Again was this a hack job or what, is not confirmed, and yes a possibility of this happening is there.

If this was a hack job, then there is more to be concerned about “the rest of us” and our civilan (err consumer) grade IOT devices like Smart TVs and wearable technologies.

Despite the outcomes for Comcast, it’s important to put this unusual outage of Cable TV services and their call center outages into proper context and perspective. There will be Monday Morning Quarterback hangover in Philly tomorrow and I would suspect there will be damage control. But please wether you love Comcast or hate them, today isn’t the day to complain. Coming from someone who understands the telecom technologies.

Phones @ Work – Local Walgreens

IMG_2485

This was a recent photo of a Cisco 7961 taken at a local Walgreens. Once a Norstar shop, I’ve been told by someone familiar with the company had cutover to Cisco company wide. Apparently since the last time I visited (which could’ve been a couple years now) they are using them.

It appears to be a Cisco Unified Communications Manager tied to some corporate data center (notice the “system message” stating “Your current options”) and if an event it can’t get through  to the data center, then you get some different message such as a fallback warning on that same screen. The paging does loudspeakers, but I’m not so sure on the sets, because the larger CM does not support set paging. (I could do a post on the lack of features in UCM 10, but then I’d start a religious war and be framed as a Cisco hater, and stuff like that.)

*

WiFi – Cell Phone Killer?

Over the last few months, I’ve been reading more and more about people using VOIP-like service such as Skype (and all the other applications I can’t name off the top of my head, must be thirty-ites a couple years early) and how that may disrupt the cellular business. Imagine people using handheld devices and not be using the public network…

It could be just as bad as “cord cutters” in the cable TV world.

I find this interesting, because WiFi is still very location specific (like a local Starbucks, stores, some apartment complexes) and the idea of municipal WiFi is still far from complete. While networks like 4G (or LTE) works somewhat like WiFi, it uses different radio waves to extend the signals (albeit less efficient than say have a CDMA stick for every 10 miles.)

I also don’t believe the death of the iPad despite what the pundits on Wall Street wants you to believe. If anything the iPad or iPod touch (and competing products) will perhaps have larger use cases for telephone service.

Personally I don’t mind this, because my cell coverage in Southern New Hampshire sucks, even on the attic level of my house (which is why I have an analog trunk from Xfinity as a fallback.) Professional business users will benefit for a few less hipster/trendy types tying up the network to post their latest meal on Instagram, etc.

I also have seen more VOIP and IP Telephony systems installed in more non profits, mum and pop shops, even my dentist had cutover to a completely new IP phone service (which resulted in new telephone numbers) the week before I visited recently. However the bugs that plagued the enterprise systems are now possibly going to plague the small users. I’ve heard one woman recently describe how they “have to reset the phone” because it doesn’t always work. (Well if you don’t have proper VLAN and DHCP settings, it’s doomed to fail, but ask them what does that do, and they’ll give you glazed eyes. )

What I mean in that case if you have networks that break down in the office, and they are already texting colleges, they more than likely will answer on their cell phone. Which of course is not good for the IP Telephone hardware manufacturers as good-to-get-by software based phones running as apps will reduce the need for a hard deskset.

Anyways, I thought I’d give some 2 cents on the topic, and by no means am I giving a definitive opinion, and I’m not endorsing ether which way, there is always opportunity somewhere in technology because there always will be people not subscribing/endorsing to the latest and greatest technology. In short, the phone guys and gals will still have a job!

Phones @ Work – Cisco 7912 – Local AC Moore

IMG_7546

This is a fine (but severely inferior) Cisco 7912 IP Phone taken at my local AC Moore store. I should say one the local AC Moore stores. This phone system has been around since the first wave of entities joining the Cisco IPT bandwagon.

AC Moore is one of those places, where it’s very location specific in terms of phone hardware. (Meaning their hardware may not be standardized throughout the company.) Another store nearby once had a Comdial system, and after a remodel it upgraded to a TDM based Panasonic system. The Salem, NH store had a Comdial (whether or not they still have, I don’t know.) I haven’t been there in years.

I would not be surprised that this is running on Cisco Call Manager Express, because if I remember, the store’s name and location is stated above the softkey area called by Cisco the “System Message” prompt. Only a CME can allow you to change the “Your current options” field, not surprised why this can be supported on the larger PBX version.

Cisco Phones Citi Branch, Midtown Manhattan

DSC_0519

 

There was tons of places of large businesses and corporations in New York which were once customers of Avaya Red and Avaya Blue or others, but many are replaced with Cisco. The only large enterprise I saw with an Avaya was the earlier picture I posted and the Rockefeller Plaza.

This is taken at a desk on a street level branch of Citibank

 

 

 

IP Phones – Mitel 5224 IP Phone

picture of a Mitel IP 5224

 

I’ve become a refugee from Avaya since the beginning of this year. I used to live and pray among their equipment, but since Avaya has gone out of touch with their user base; I decided maybe its time to really move on.

Mitel (pronounced my-tel) has made some nice IP equipment in recent years. Back about a decade ago they made some wacky designs – on their IP phones, but of the last few years they are pretty decent looking. This Mitel 5224 set has 24 buttons; 4 line appearances (similar to the 3 rollover lines that Avaya refers to call appearances) however once you go over 4, then expect to hear busy signals; and the rest can be used for speed dial and depending on the setup you could use BLF. I do not have a proprietary Mitel PBX, so I am using the SIP firmware.

The set is well designed, the handset is wideband, and the rubber on the handset is great for a built in shoulder rest. The LED on the top serves as the message waiting and the LED blinking pattern goes in sync to its 24 ring tones, really cool. One of the quirks (actually not Mitel specific) the volume key actually doesn’t change the ring volume, you have to go into menus like a cell phone to change it, as it only changes the contrast when idle. Another problem, again not just Mitel is the requirement to press the dial softkey to initiate a call. This is one of the major setbacks of using SIP, it works somewhat like a phone, but there are these kinds of setbacks.

I do love the LED backlit display, which makes it friendly in dark areas or get your attention if a you missed a call.

Implementing Asterisk, part two

This second installment as an ongoing project implementing Basic IP voice services though the Asterisk system, I’m going to walk through the keys to reliability. This is mostly a hardware discussion.

Single Vendor, Common Model = Limited Glitches

In any moderate to heavy VOIP setup, it should be recommended you stick to a single vendor/models. Try different sets to see what works and what doesn’t work. I personally lothe Polycom outside of their analog offerings. I don’t like their design (hate how the hand/headset cables just sit out (pleases them IT layabouts I bet). and while they are cheaper than Cisco, its essentially a Cisco clone because the phones have lots of limitations in my opinion.

Nortel might be a decent solution if you want to try using UNISTIM, not sure if that would be a CPU intensive process or not. I’ve read that Avaya IP Phones (outside the proprietary 4606/12/24/30 models) can run behind an Asterisk if you have it on SIP mode, but had limited success with my attempts.

Cisco would be a hit or miss, and would be a miss you can’t decode the code-heavy setup of Cisco. There are third party plugins like chan_skinny, but again it can be time consuming. Also if you love Cisco, I recommend using the Skinny protocol. Why? You have wideband audio, especially for the speakerphone (SIP is just as bad as analog voice), the sidecars do not work on SIP (maybe in newer releases, not sure) and other advantages than the trial and error setup of SIP.

Labs or hobbyist with one desk having a Cisco another one running Grandstream and the bedroom having Avaya is one thing. Trying to run them in some production or a business where voice is critical can be actually be unprofessional. It could break a uniform look and managing them can be horrific. DHCP services vary model to model.  As mentioned in the first part, I’m using Mitel in this project, which requires Mitel-specific DHCP options. Some IP phones limit the use of TFTP as its becoming more antiquated due its insecurity. Mitel, and Avaya have the option to use HTTPS or HTTP server, Cisco still is stubborn with TFTP, even when its now deemed an unsafe solution. Make reference of that. Some phones Avaya’s IP Telephones/Deskphones commonly requires using a .txt file to receive settings, known as the 46xxsettings.txt (referring to the first generation IP phones that Avaya made.)  Cisco has very pretentious ways of deploying if you don’t know how to configure a Cisco phone outside the protected gatekeeper, the CallManager. Unless you know UNIX and how to code, and write foreign files, then this can be for you.

Networking Skills Required

Another laugh out loud moment is how many people who set these things up probably for a bragging purpose often have to learn new things like TCP/IP, and other services that don’t get mentioned on the sites of the distros. In order to have a happy and healthy life running VOIP/IPT in production, you will need to learn basic to moderate understanding of how QOS (quality of service) and COS (class of service) as well as VLAN. I don’t have the time to discuss all of these, so it’s best to at least do other research online and with books on these subjects

SIP = Potential Resource Hog

As mentioned in Part 1, SIP can be a resource hog because it’s an app that puts strain  mostly to hardware. Ensure you have your Asterisk has enough resources to handle the volume of calls. Again if its a lab  or at home and all you need is 2 concurrent calls you might get away with, but if you are an enterprise (over 200 terminals) – you’ll need to do homework and proper planning. SIP doesn’t tie up a network if properly planned, but can cause problems on the “switching” end (in this case it would be the server running it.)

What can happen?  The phone might get locked up for about 10 seconds getting a call to go into place (this can be a serious quality problem.) Sometimes you might go on and off hook because one looses patience getting a call appearance to fire up. (File this under the  I Press the Windows Key and it does nothing, and the Start Menu goes off and on for 20 seconds.)

Also ensure the PC has enough RAM and hard drive space to run the application, bears repeating.

Single PC for a Single Application (Separate Server for an Asterisk)

This can’t be stressed enough! There are going to be whack jobs who think they can run their PBX on a same system that does file sharing or even worse a domain server.  This is what really drives me nuts with IT guys who want everything open, then they start bitching why there are so many attacks on their networks!

After installing the Asterisk distro, typically after logging into the system via the CLI prompt, type SETUP and it should take you into a Linux screen setting up TCP/IP, firewall and DNS settings. Check the only things like IP, DNS, TFTP, FTP, DHCP, HTTP, and HTTPS. Anything not relevant for the use of IP Telephony, TURN OFF!

Macs running a minimum of Mac OS X Tiger can possibly run Asterisk. If you are someone who is a visual user, or needs a graphical orientation of Asterisk, I would recommend this. One catch. You should again run this Mac that will be dedicated to do telephony and remove all non-essential apps (except for the ones in the Utilities folder and apps like the Text editor I forgot its name off my head. A Mac running on a gig RAM and about 80 gigs HD can set up a sub 20 user environment. I have not done this, but I have spare Macs that can run OS X, and this could be a follow story when time warrants.

Redundancy  

Redundancy is not documented, at least from what I can read. I do not know how to create an Asterisk as a redundant server, therefore I can’t document this at this moment. Redundancy is important to any software based PBX system. Avaya’s Communication Manger and Cisco’s Unified Call Manager as well as major players offer (and some require) at least one or more redundant servers. If there is a solution, I’ll post it

Separate Servers for Asterisk Services/Applications

Another “gotcha” (at least with my experience) is because the Asterisk systems are so integrated, if you got a single PC or server, it may not be able to handle all the applications as one.  Again the guy who invented Asterisk was allegedly modeling a “PBX” from ether a Norstar or a Merlin Magix, which runs a virtual “PBX mode”. Both systems support up to 200 ports, and again you’d be very lucky to run at least 100 without problems on a lower spec PC. Again this may bring up problems if you do not have enough resources to run on a single box.

SIP Gateways

Gateways typically take the packet (IP) data and convert it to another medium. There are data gateways that convert data to a DSL medium, to cable (coax),  T1, etc. In VOIP there are similar gateways that converts the voice pakets  into another medium, that can be for analog trunking, T1, PRI or even to another PBX. For this exercise, I will be using analog called Foreign Exchange Office or FXO.

Lets focus on PCI cards. A single T1 card might work, or one or two PCI cards for Plain Ol Telephones (POT) or analog trunking. Some VOIP setups can be strictly analog for phones and one IP phone for attendants, and the trunking could be VOIP based. Before setting the foundation for trunking or analog phones or other gateways start planning that first. Typically most workstation class PCs have up to 6 PCI slots.  Another “gotcha” moment would be in this case if you are wanting to do a single box setup and all the hardware in one box plus everything else. This can put the system into high stress and cause the system to crash or cause calls to hang or not be answered.

Now if that is the case, I suggest looking into small appliances like FXO and FXS gateways. Just like a router or a wireless access point (WAP), these little to big guys won’t be chained into the PC’s logic board and are more mobile. You can have your communications server in the data center and have your analog hookups in the area they are most needed in and then plug them into the LAN via an Ethernet cable. This reduces the cost to rewire or add wiring from the remote wiring closets to reroute to the data center or vice versa and reducing the need for the PC to handle gateways so it can focus on routing calls, or taking voice mail.

Gateways – Location Dependent

It’s important to understand gateways need to be in locations where they are needed the most. Marketing materials show the server next to the gateways, and while some environments may work that way, don’t believe the PR entirely – another IT mistake! Gateways were once called a processor network or port network in the pre-IP days. PNs were located in wiring closets where the phones were located. If you had remote sites in a campus environment or if you are in a hi rise building, you’d have these things in different parts of the building or land. Before the transition to IP most phones could travel up to 3,000 feet  (you miss those good ol’ TDM days?)

Physical environments are not virtual, IP is. You can have gateways in any part of a building and have the sever in a totally another location, only catch is to ensure you have that documented in case of difficulty.

Part three coming soon with more scope on networking side of things.