Camvex, Sony Warehouse Network

24 Aug 2010
by: John Adams
Camvex has designed and commissioned a networked video surveillance system for The Reject Shop’s new distribution centre in Ipswich. The system incorporates 33 Sony IP and megapixel cameras, as well as servers running Sony’s RSM software.

FOUNDED by Ron Hall and John Shuster in 1981, The Reject Shop might seem like small fry on the retail scene but recent history tells a far different story. The Reject Shop is a seriously dynamic business. In 2007, the company hit the ASX Top 300 and in 2009 it opened 23 new stores in just 22 weeks. As part of The Reject Shop’s massive expansion, the company opened a new distribution centre in Ipswich earlier this year to service more than 90 stores in NSW and Queensland.

 

The state-of-the-art 26,600 square metre Ipswich distribution centre cost $A16 million to build and as part of the fit-out, The Reject Shop management decided to leverage the site’s new fibre network and incorporate an IP video surveillance solution to provide local and remote monitoring and recording.

 

According to Darren O’Connor, The Reject Shop’s Chief Information Officer, the networked surveillance system at The Reject Shop’s Distribution Centre is designed to give the company a fully integrated solution that allows images of activities in and around the facility to be recorded. The solution is designed to be simple, robust, affordable, proactive in operation and accessible from remote locations for monitoring and maintenance.

 

From a purely operational perspective, The Reject Shop has a number of specific requirements, including being able to read license plates of vehicles coming onto the site and being able to track the movement of vehicles around the site. Along with these key elements, the system is also used for process control and allows management to hone the efficiency of the facility’s 6000 carton-per-hour output.

 

“The surveillance system is primarily designed to enhance process control and ensure we have an efficient operation,” O’Connor explains. “Of course, that focus on efficiency also applies to the surveillance system itself, including a design that leverages existing infrastructure to allow easy operation and remote maintenance.

 

“Also important to us was meeting our key criteria without spending money needlessly by exceeding them.”

 

According to O’Connor, The Reject Shop’s decision to go with an IP solution was governed in great part by improvements in technology over the past few years.

 

“We wanted a particular network architecture and we wanted a particular compression algorithm and there were very few products on the market that gave us all the things we wanted,” O’Connor explains.

 

“A huge driver for going IP was having a straightforward integration to the rest of our network and to related sub systems rather than the CCTV system being an island on its own,” O’Connor says.

“In the past there was no easy way of integrating these systems at a reasonable cost. So from our perspective it’s really only been in the last year-and-a-half where we have seen the convergence of capability and price that allows the affordable integration of multiple security systems.

 

“Other things that encouraged us to go IP were cameras we could power using PoE switches, the ability of cameras to be displayed on monitors with integrated PC control and compression good enough to allow us to send multiple IP video signals over our wide area network. In the past we could not do these things without significant compromises in frame rate, quality or cost.”

 

O’Connor says the ability to record and index events now is so much easier than it was using 5-year-old technology.

 

“Even with DVRs you had to sit through hours of video to find something, or you were relying on time stamps as an index,” he says. “But the current management systems make events infinitely easier to find – it’s a huge benefit to end users.”  

 

According to O’Connor, the system gives staff a strong understanding of how things work in the DC.

 

“It’s hard to get mental models built up in people’s minds if they have not seen or cannot remember the layout of the site,” he explains. “The site is a couple of thousand kilometres away so you can’t just go and visit. But with the surveillance system we can see it immediately.

 

“In the future the system may be used to allow us to change the site and to appreciate the way things are laid out as we plan for a redesign. You can look at drawings but it’s easier if you can view the site and see the physical constraints it imposes.”

 

Another major issue for O’Connor and The Reject Shop is efficiency of the surveillance system itself.

 

“In IT we don’t have a lot of arms and legs in the field,” O’Connor explains. “We are a chain store and we operate using what we call ‘centralised direction, local execution’.

 

“There’s no field force working for IT so we rely on the arms and legs of suppliers to go and fix a fault. We have to cover all of Australia and would need a small army of people to do it. As a result we rely on smart systems and the support of suppliers.”

 

According to O’Connor, this translates to using the CCTV system to direct suppliers to where they need to be in real time. There’s even a plan afoot to install a PTZ in the DC’s network room specifically to allow management to assist suppliers get vital maintenance done.

 

“In the computer room up there it’s about trying to direct people to what you need them to sort out and while things are labelled, we have thought of a PTZ in the room so we can say to contractors: “Yes, that’s it there – the rack you have your hand on”.”

 

“Like the rest of the cameras at the Distribution Centre, that PTZ will help us manage the business more effectively. It’s hard to deal with the sheer size of Australia,” O’Connor says. “You need systems that support remote diagnostics, support remote analysis – that can feed you information. Modern CCTV systems need to offer this functionality.”

 

The Reject Shop solution

 

In terms of components, the system in the DC incorporates Sony SNCRX530PB IP PTZs and fixed SNCDM160 megapixel cameras supported by Sony Real Shot Manager Software running on quality off-the-shelf servers. There’s also a Sony Real Shot Client on a workstation on site, as well as PoE switchers, routers and UPS gear supporting the RSM servers. Additional local monitoring is handled by a 40-inch vertically mounted LCD screen, while other key pieces of hardware include a pair of Barionet IP Automation Controllers which allow alarm inputs to be carried onto the network.

 

There are 33 IP cameras in total – 20 are megapixel – and the others are IP PTZs running at 640 x 480 VGA. All are dual stream, with the JPEG stream recorded to the 2 RSM servers (1 master, 1 slave) and recording distributed between the servers based on how much HDD space particular cameras are likely to need.

 

“In terms of system structure at the Distribution Centre we have a number of fibre rings that run around the DC – both inside and outside,” explains O’Connor. “Those rings are bi-directional – they can work forwards or backwards so if there’s a failure there’s redundancy.

 

“We have active gear distributed throughout the building – so there are switches that can power the equipment. There are intermediate communications racks and that’s where the Cisco Switches are. Then there’s a comms room. The monitor signals come direct from the comms room, as does the link to the Sony Real Shot Client in the manager’s office.”

 

According to O’Connor, the way the system works is that IP video data signals from cameras connect back to data cabinets containing Cisco switches.

 

“These switches connect to the fibre backbone which comes back to the comms room and where the fibre connects to the main layer 3 switch. IP video data streams from there to the Sony RSM Servers, which have Sony’s RSM software installed on them, as well as on demand to the monitor and manager’s workstation.” he says.

 

“Each of these RSM servers are supported by UPS to ensure redundancy. The RSM servers have the operating system and the RSM software installed on solid state drives. There are a number of removable HDDs in each server and camera data is recorded to these. The units have 6TB each – that’s 4 x 1.5TB HDDs.”

 

O’Connor says The Reject Shop has a 4Mbps MPLS network with Telstra for remote access.

 

“Using this network we can connect to the Distribution Centre servers from our office here in Melbourne,” he says. “We run some encryption over every link to protect ourselves. That 4Mb network is suitable for running a number of cameras simultaneously. We also have the ability for these cameras to multicast and our network is multicast capable, everywhere.”

 

The system at the Reject Shop was designed and supplied by integrator Camvex and the company’s CCTV systems engineer, Andrew Del Biondo, says a driving force in the overall layout is that the system is not designed to be always monitored in real time.

 

“Images can be viewed in the Kensington office in Melbourne or they can be viewed in Ipswich on a monitor or workstation,” explains Del Biondo.

 

“From an onsite monitoring perspective the manager of the site has a workstation that’s dedicated to the system and there are a couple of monitors connected to it so he can view cameras.

 

“Our web designer, Glenn Conway, has also designed an interactive map that’s integrated into the Sony RSM workstation and the manager can click on an icon and view a camera and do playback.

 

“However, live monitoring is not the main purpose of this installation,” Del Biondo says. “Instead the Reject Shop records cameras and if there is an incident, the images are viewed. Management will also use cameras to see if there are ways to improve efficiency on the site.”

 

According to Del Biondo, the Sony RSM software used by The Reject Shop is a nice mid-range software product and ideal for a medium-sized site like The Reject Shop.

 

“Because it’s made by Sony the software works with the cameras and all the camera functions work as you expect them to,” he explains. “One of the risks with 3rd party integration is that when you bring together multi vendor products and software including cameras, a VMS, network hardware and firmware, there are many variables and sometimes the different suppliers blame each other when things don’t work properly. We didn’t have a worry with that in this installation.”

 

The physical installation

 

Prior to this install, Camvex had worked with The Reject Shop on a number of other projects including designing and installing CCTV systems for some of The Reject Shop’s retail outlets. This relationship meant the two companies had a good understanding of each other before the DC system was installed.

 

“First we spoke about installing a surveillance solution in the Distribution Centre and we undertook a design based on my understanding of what the loss prevention department wanted from a security and safety point of view, combined with what Darren considered would be appropriate and suitable for the Distribution Centre as far as a networked solution was concerned.”

 

Del Biondo says that in terms of the physical installation, The Reject Shop’s construction contractors were in the midst of building the Distribution Centre when Camvex got involved.

 

“By the time we were awarded the contract there was an alarm installer already on the site and we recommended The Reject Shop request the alarm contractor install the camera cable or use their own data cabler which would reduce costs.

 

“We then supplied the cameras to the client for the alarm contractor, Ambush Security, to install,” Del Biondo explains. “Once the cameras were installed we did the commissioning with the whole installation running over a couple of weeks.”

 

Perimeters at the DC are about 200m and there are no runs exceeding 70 metres and this meant there was no additional complexity to the network cabling than would be encountered with any edge device.

 

“Interesting from our perspective is that the builder did a tender on an analogue CCTV system and it was more expensive in this particular application,” Del Biondo says. “This was due to the fact the network at The Reject Shop Distribution Centre is excellent and can handle megapixel resolution and data easily.” 

 

At the DC the Sony megapixel cameras are all 1.3MP and they are running at full resolution.

 

“We find 1.3MP is a nice sweet spot,” says Del Biondo. “We are running all the megapixel cameras at full resolution in JPEG compression, 1280 x 960, 3 images per second. When there’s an alarm event this frame rate is kicked up to 6 images per second.” 

 

According to Del Biondo, Camvex techs built the RSM servers from scratch using off the shelf components and this allowed them to improve the specification while reducing costs.

 

“We built the 2 servers and to improve their reliability the Windows operating system and the Sony RSM software are installed on a non-volatile 32GB SSD,” he explains.

 

“Cameras are recorded to the onboard HDDs with a total of 6TB per server and it’s a very simple solution that’s very easy to support. Because the servers are uncomplicated, if a hard drive or motherboard fails it’s like repairing a PC. And it’s low cost compared to some other solutions, as well as being very flexible and economical in terms of enhancing storage.”

 

Camvex also installed a 40-inch Samsung LCD/PC monitor to allow staff to check vehicle entry points.

 

“We mounted this vertically because The Reject Shop wanted a permanent display of 3 specific cameras for live monitoring but did not want anyone to be able to change the camera display,” Del Biondo says.

 

“There is a PC built into this monitor but there is no client software supporting this display. Instead we are accessing the cameras via a web browser we have customised to display those cameras. It displays better vertically than horizontally. This is independent of the RSM and gets streamed through Quicktime.”

 

And another key element of the installation was putting in the Barionet Controllers which handle integration of the alarm system and the surveillance solution.

 

“We used one Barionet device that has 8 inputs and one that has 4 inputs,” says Del Biondo. “The Barionets were installed close to the alarm panel so this allowed very easy access to the network. In terms of configuration you make the association between a Barionet input and a camera through the RSM software and then you set up a number of actions. It’s so easy.”

 

For O’Connor, a key element of the physical installation was maximising use of existing infrastructure without impacting on data or VoIP – both of which share the network at the DC. The advantage for The Reject Shop was a reduction in installation cost.

 

“With analogue the cameras themselves are less expensive but when you take in the installation of coax cable runs on a site this size you are up for a fair amount of money in cabling costs alone,” O’Connor explains.

 

“Conversely, if you are going to leverage existing infrastructure with the only change required being going to Gigabit instead of fast Ethernet, then it costs a little bit more but still far less than analogue. And you have a superior network. At the DC the fibre rings running at Gigabit speed and we just connect cameras into the nearest network point. It’s completely future proof.”

 

Alarm integration

 

If the mere thought of integrating alarms and video surveillance suggests complexity, think again. The nature of the integration at The Reject Shop is simple and in some ways it represents a snapshot of the way all systems might be integrated in future. It really is very well done.

Del Biondo says the thing he particularly likes about this system is its neat integration of the CCTV solution and the alarm system, and it’s impossible not to agree with him.

 

“The Reject Shop wanted to integrate CCTV with their alarm system – they wanted PTZs to swing to presets when an alarm event such as a door opening occurred,” Del Biondo explains.

 

“Of course, there are multiple ways to do this, you have High Level Interfaces or Low Level Interfaces and in many cases these integrations are expensive and complicated. But with this system, all the alarm installer had to do was put in his relay boards as usual and then give us an output for the relevant doors.”

 

Integration from alarm panel to the network could hardly be simpler.

 

“We just connected these outputs to a Barionet 50 – an IP Automation Controller with Modbus/TCP and SNMP support,” Del Biondo says. “The way this unit works is that you connect your alarm to the Barionet’s I/O inputs and it ports them to the network via RJ45 and sends the alarm signal via IP.”

 

Importantly, each output is associated with a particular camera and can be used to tell a camera to go to a particular preset. According to Del Biondo, it’s a solution that’s elegant in its simplicity and extremely affordable.

 

“A Barionet costs a couple of hundred dollars and eliminates the need for a high level interface, making integration so much easier,” he says. “It doesn’t matter which alarm system or access control system the customer has – they give us an output and the Barionet will convert this signal to IP and it can be associated with any Sony camera you want via the RSM software.”

 

On the management side, the alarm inputs are tagged to particular cameras and Sony’s RSM software has a number of recording modes that can run simultaneously.

 

“In this application we record continuously at about 3 images per second and we then have alarm recording based on motion detection from the camera which kicks the frame rate up to 6 frames,” Del Biondo explains.

 

“We also have event recording which occurs when we get a signal from the alarm system – it also kicks recording up to 6 frames per second.

 

“The fact the Sony RSM integrates with the Barionet devices made life so easy for us. What the Reject Shop liked about it was that there was a clear line between where the alarm providers responsibility stopped and ours started. And we loved the fact that getting the Barionets onto the network was very easy.”

 

Network and system management

 

Because the system is monitored remotely, the WAN a vital element of the mangagement solution and when moving video over the MPLS network for viewing, the design exploits the MPEG-4 stream of each camera, with video streams each having a bandwidth of 2Mb at 3 images per second.

 

Bringing all the surveillance hardware together over the network is Sony’s RealShot Manager (RSM) Advanced Version which incorporates a bunch of capabilities that make it a competent mid-range management solution. One of the nice features of RSM is that it incorporates intelligent video analytics using Sony’s Video Motion Filter Alarms and Video Motion Filter Searches features. These use metadata to provide operational efficiency and a high level of security.

 

There are also Video Motion Filter Alarms that allow users to define parameters and to fine tune alarm triggers for live monitoring and recording so that crucial events are easily called to the attention of a guard and are recorded. Video Motion Filter Searches allow users to define search parameters in situations so events of interest are easily searched for in recorded information.

 

The Reject Shop has not applied video motion detection to its new system yet but Del Biondo says this is being considered as an option for the future. In the meantime the video management features of RSM make the system easy to operate.

 

In terms of the workstation viewing screen, the RSM combines a site map complete with active icons on the left with a viewing window on the right. There are also coloured time lines and all the usual video play buttons. According to Del Biondo the timelines make the system very easy to use.

 

“When you go into the search module of the RSM software you can ask the system to display all those files or nominate the ones you want to see,” he explains. “The display shows the different recordings in different colours in the form of time lines – continuous recording is green – red indicates motion and orange relates to event recordings.

 

“This makes it much easier to search and I think with camera systems if you can make it easier for a customer to find things, that’s far better. Continuous recording is the most reliable method but it uses a lot of storage so if you run a combination you get a better outcome.”

 

“In terms of live monitoring, as well as standard camera views there are also a number of presets you can pull up – entry/exit points and vehicle access points.”

 

Up in Ipswich, what is exposed to end users is fixed. According to O’Connor, when it’s turned on it the LCD screen  automatically links to 3 camera presets. The primary use of these camera views is to view and recognise vehicles coming onto the site.

 

“We went with the screen and embedded PC solution because if you install a PC, people treat it like a PC – they reboot it, mess around with it, kick it under the desk,” O’Connor says. “If you make this interface look like something else even if it is a PC, they treat it differently.

 

“It’s proven to be an excellent solution,” he says. “The screen shuts down at night and comes on in the morning. It’s capable of 24x7 operation and handles its own cooling and power management. It takes up very little space and is visible to everyone in the room. The screen is also very clear and it doesn’t need anybody to do anything – it’s perfect.”

 

But where management is concerned it’s the overall system’s ability to be managed using a WAN that makes it ideal for The Reject Shop.

 

“The remote management features are comprehensive,” says O’Connor. “The final configuration of the cameras was done from here in Melbourne – setting the exposure levels, getting the presets right.

 

“Essentially the nature of the CCTV system mirrors the use-parameters of the system itself. It’s the same focus on efficiency. We don’t want the costs of sending techs up to Qld for system maintenance and because of this, Camvex needs a simple footprint on the ground to make it easy for them to manage things remotely. It makes so much sense if we can work in this way.”

 

Because the system is being monitored and maintained remotely, the health of the network component is also important.

 

“On the network side edge devices are managed using IBM Director – that includes monitoring for hardware faults, disk failure and problem predictions,” O’Connor explains. “For network monitoring we use Cacti, which is an open source networking monitoring solution. It allows us to poll the environment every 5 minutes and actually see video flowing around the building at a data network level.

 

According to O’Connor, most alerts go to The Reject Shop’s people but there are suppliers who also get reports and he says that’s how all IP-based security solutions will be supported in the future.

Del Biondo agrees.

 

“One thing Darren is organising is giving Camvex remote access to the system in order to handle system maintenance and support,” he explains.

“The way this will work is that instead of connecting directly to the system up in Ipswich, we will connect to a desktop here in Melbourne that’s got direct access in order to keep bandwidth demands down.”

 

But there are some other issues to take into account with a megapixel-rich CCTV system running on a shared data network.

 

“The image streams at the DC are on a VLAN in the distribution centre and we keep a close eye on them – we have to be able to identify traffic for QoS on the network,” O’Connor explains.

 

“This is a shared network and I can’t have video streams interrupting phone calls. We recognise the fact that bulk video transfer has a significant impact on our network and if you’re serious about video on shared networks you have to be prepared to shape it into the network by dropping packets or making some other choice to reduce network load if required.

 

“We track video as soon as it leaves the camera so we can actively manage traffic. Having said this, we have no problems because the Distribution Centre has significant bandwidth,” O’Connor says. “The most over-used link would be around 20-30 per cent utilised so there is plenty of headroom left.”

 

There’s a lesson for other security managers and IT managers in the methods employed by The Reject Shop. This is a true shared network with significant core uses.

 

“Both the business use and the surveillance use of the network are QoS rated and we still prioritise data in order to guarantee refresh rates. It’s a busy network. We have a lot of radio gear including body worn units, vehicle terminals, hand held terminals, cart-mounted Panasonic tough books and local network connections for PCs,” O’Connor says.

 

“There are also more than 30 printers but the network is smart enough to work out what’s what and to manage our needs. We are comfortable having all this data sharing our network as long as we can treat it appropriately.”

 

Conclusion

 

It’s clear from the start that The Reject Shop is pleased with its networked video solution.

 

“We went down the IP Video path to get a fully integrated solution that would allow us to really capture what was going on inside the Distribution Centre. We did not want to gamble on cameras always pointing in the right direction, we wanted to leverage presets and to have a clear demarcation between alarm providers and the security system with the Barionet devices,” O’Connor says.

 

“With this system we can see what happens on the site – it’s very clear, very simple and very cost effective. It’s just such a basic integration. In comparison, high level interfaces are fraught with danger, with software incompatibilities, with upgrades.

 

“They also have the potential to dramatically overrun costs for not a lot of value,” O’Connor explains. “This is different – the alarm installer can deal with the I/O requirements and on the other side the Barionet module integrates into the Sony RSM and allows you to pick up events and drive cameras to where you need them to be.

 

“Because it’s simple technology you are more likely to be able to rely on it. When Camvex first suggested this as a simple layer between the alarms and the CCTV it appealed as a really simple way to resolve the issues of integration while allowing us to focus on what we wanted to achieve and we are very happy with the results.”