Take stake overflows as an example. Harmful as they are in potentially crashing a system, an equally severe issue is that they lay the foundation for a security vulnerability. By exploiting a stack overflow, an intruder can execute instructions in the software that are possible only after an authentication.

The following is a simple C program that simulates the authentication logic of a device.

#define PASSWORD “secret”

main (int argc, char **argv)
{
  if (1 == password(argv[1]))
  {
    printf(“Congratulations, right password\n”);
    /*
    * User authenticaed, do work here
    */
    return;
  }

  printf(“Access Denied, wrong password!\n”);
  /*
  * User login unsuccessful!
  * Exit
  */
}

The password() function is as follows:

int password (char *buf)
{
  char var[16];
  strcpy(var,buf);
  if (password_compare(var, PASSWORD))
    return 1;

  return 0;
}

To exploit the potential overflow in the 16-character ‘var[]‘ buffer, on a 32-bit system, one can simply pass 20 characters followed by the address of any instruction in the program. This overflows the stack, bypasses the authentication check and gives the attacker access.

If a similar piece of code was part of the authentication logic for a command line interface (CLI), your network system is at a very high risk of being severely compromised.

Fortunately, as developers, without having to be security experts, we have ways to mitigate such risks. Automated code analysis can allow you to take the guesswork out of finding security problems in code while you focus on adding features and functionality. Defects such as resource leaks, uninitialized variables, and memory management problems can all serve as a basis for eventual vulnerabilities such as unauthorized access, arbitrary code execution and denial of service. Automated analysis using static analysis checkers that find the resource leaks, uninitialized variables, buffer overflows and much more, should be a part of the development process if we care about security within the telecommunications and networking systems we build.

Recently Matt Hayward from Coverity and Todd Morley, Product Architect at Pitney Bowes, hosted a webinar discussing the business value of software quality. Whether you realize it or not, common programming mistakes – errors, bugs, defects, whatever you call it – have a dollar-value, a business value associated with them that cannot be ignored.

There is an expectation of greater reliability of programs we create and the languages in which they are expressed. The languages have evolved to a certain extent, with Java and more recently C#. However, there are enough elements in all languages that are potential billion dollar mistakes. It could be the buffer overflow, the memory corruption, the resource leak or something that might on surface is just a simple uninitialized variable.

When we start thinking about the impact a single software glitch might have on the mobile phone that the software runs on, or the embedded system that is controlled by the software, or on that networking device that manages data across networks, we start to see how one defect can translate to a high cost for a business, both in dollars and public image.

Take the case of Motorola and Nextel Communications. A software glitch in Motorola cell phones left millions of Nextel customers without a safety feature that gave 911 operators their precise location. The mishap caused Nextel to miss a year-end 2005 deadline, set by the Federal Communications Commission (FCC), to equip 95% of its phones with technology that lets emergency workers pinpoint the location of customers who make 911 calls. More importantly, the business failure as a result of this glitch didn’t help Motorola’s effort to mend relationship with Nextel and was a big blow to Motorola’s effort to differentiate its product from their competitors.

Fortunately, there are solutions and technology that help. The key is static analysis, which needs to be integrated into existing development process and proactively utilized as part of the development cycle in order to prevent these potential billion-dollar defects.

First, it’s high time there was real competition in this space. Coverity is now the largest standalone static analysis company, and we are best of breed in every sense: product, services, and ability to execute and deliver to even the most demanding customers. This news should open up an opportunity for customers to reconsider what they should expect of a static analysis security vendor.

Second, I predict this will be the end of the period when security teams try to use tools to hammer on development to fix issues. This purchase shows security for what it is – a step that will be integrated into the application lifecycle process. Fortunately, this is a perspective that Coverity understands. We know what it takes to get defects actually fixed, not just found. We also have a vendor-neutral approach to integration with third party tools, and that matters in the heterogeneous environments most enterprises have.

Coverity has always been dedicated to providing the most accurate and actionable static analysis products on the market. Developers must end up adopting static analysis tools, because only developers can fix the software. This fact won’t change no matter how many tools are marketed and sold to security teams.

That’s why we partnered with Armorize on web application security. Armorize shares this vision of getting security into development in a natural way. Combined with Coverity’s static analysis for quality issues, we bring the market something it really needs: software security analysis that actually works for developers.

More on this later – but for now, read more in this press release.

Source: http://www.osbr.ca/ojs/index.php/osbr/article/view/610/571

In a recent discussion with VDC analyst Christopher Rommel, we covered challenges in software for telecommunications and networking industry and the effective solutions. An area that often gets overlooked when discussing networking system software issues is security vulnerabilities. This probably has to do with comfort in knowing that the various layers of the network stack present barriers to the intruder. That might be a misguided sense of safety.

To start off, an intruder needs an access point, a connection end point that is reachable over the network. Most telecom/networking devices have a Command Line Interface (CLI) access through a telnet port or a secured SSH port. Very often, these are open ports that are reachable through an IP address that is not behind a VPN or is accessible without a secure tunnel. Even devices that are within the corporate network, such as load balancers, firewalls and WAN optimization appliances, have a reachable IP address that SNMP management clients can get to.

Once a connection point is identified, the exploit is usually related to a stack buffer overflow. By exploiting a buffer overflow in the authentication logic for example, an intruder can execute program instructions without a valid login. Through malicious user inputs and CLI commands, it is possible to cause a stack buffer overflow that results in root access on the device. At the very least, by exploiting a buffer overflow and crashing the system, one can cause a denial of service attack.

One other area of weakness is in use of temporary files on the system to store critical data. If temporary files are used to store logins, a hacker can exploit that weakness to get to and misconfigure the device. The magnitude of such an attack is potentially very large. For example, one can configure a router such that all data-plane traffic (data that the application is communicating between the server and the client application) is routed to insecure and hacker-compromised devices. With the shift to the Software-as-a-Service model for business-critical applications, the integrity of data going through the networking and telecommunications devices is even more essential.

The challenge in preventing such vulnerabilities in C/C++, the languages of choice for most networking systems, is due to some of the language’s features. As discussed in this article on language insecurity in Open Source Business Resource, these are:

• Lack of type safety
  • Execution of C/C++ programs does not stop as a result of an error making unauthorized access possible.
• ” Pointer arithmetic
  • o Programmers can change the value of a pointer, and thus read and write anywhere in the process memory space making arbitrary execution of code possible.
• ” Static buffers
  • o Such buffers do not grow and the access to buffers is not checked for bounds resulting in overflows and overwrites of memory regions.
• ” Lack of robust string type
  • o C has no native type for character strings, meaning static buffers with potential overflow problems are used instead. In C++, programs can use a String type, though the usage is not very often.

An audit of software at various layers of the network stack is essential to ensure that devices and systems handling enormous amounts of data are not compromised at any layer.

Telecommunications and networking devices are designed with a strong emphasis on redundancy. On top of that, the network architectures created using these systems are designed with multiple paths and redundant links.

This has an interesting implication on the software running on these systems. When software fails due to defects in code, it doesn’t make the news. At least it doesn’t make the news like the latest security breach in a browser or software problem in a car.

However, can no news of software failures be assumed to be good news? Dig a bit deeper into the system logs and you will get some answers.

The major issue is that network architects are forced to design by taking failures into account. This not only increases the complexity of the networks, but financially this means higher costs for equipment, management, maintenance and services.

The systems are designed to minimize downtime through two basic approaches:

• Recoverable systems – No human intervention, but downtime and outage till the problem is rectified.
• Highly available systems – Keep running even during failures with hot backups.

For recoverable systems, the main issue will be the downtime with the impact being that the systems are unable to meet the 99,999% (“five nines”) high availability SLAs. The highly available systems on the other hand require replicating information which is directly proportional to cost and complexity.

The best solution is then to ensure the integrity of software that runs on these systems. Ensure that code is defect-free; avoid crash causing errors such as illegal memory accesses, null pointer dereferences, performance degrading problems such as memory leaks and prevent security vulnerabilities through buffer overflows and insecure handling of data.

We recently discussed the value of effective software analysis for telecommunications systems based on the current development trends in complex telecommunications and networking systems.

For software failures in telecommunications systems, no news is not good news; it probably just means that sooner or later, it will be news.

Back in 2001, I worked at a telecom company in the routing software group. Besides the fact that I was writing code that would be running in systems in the Internet backbone, the most enjoyable part were my lunches with Tom where he shared war stories from his illustrious career. Most of Tom’s stories began with, “Back in those days…“.

Tom was a veteran software developer. More importantly though, Tom was the best software developer I have worked with. What made him best was not just his knowledge and skills, but also a knack for quickly adopting and integrating the latest software development techniques into his work. His ability to adopt and integrate new ideas and methods has always allowed him to be efficient and excel at what he does.

Evolution

We unleashed Coverity 5 this week. When discussing the salient features of this version of the product with a potential customer, I thought of Tom and the evolution of software development techniques.

As software professionals, our challenges are vast. Delivering stable, scalable, high-performing mission-critical products under ever-shrinking schedules is what we need to do day-in and day-out.

On our side, fortunately, is the evolution of software development tools and technologies. We are not in the stone age of software development. Single file text editors have evolved into feature-rich IDEs. Simple functional programming has developed into object-oriented methodologies. Templates in the C++ standard library have made it unnecessary to rewrite functions and classes for every data type in your code.

The Coverity 5 solution suite is an example of the evolution in the technology to achieve software integrity. Features such as deep analysis that finds critical defects, intuitive code navigation, and project impact mapping, allows developers to easily find and understand defects and their impact to all projects and products sharing the same code. Development managers now have a way to understand the business context for prioritization and remediation of defects based on impact to the business.

These are features that allow us to be efficient in what we do well — write software that adds functionality, design and code algorithms that are optimal and deliver stable, scalable, high-performing mission-critical products. Tom would adopt this in a heartbeat.

Exciting Times

As we face new, unique and large challenges in software development, we’re also very lucky that there are new and emerging technologies to lean on. Quality, simplicity and ease-of-use are features that our customers expected us to deliver. We demand the same from the technologies we adopt. These are indeed exciting times to be in software development.

Why does this happen? The pain caused by software defects are mostly outside of the development organization – downstream there is QA, docs, support people, field representatives, and much more. When something goes wrong, these people form a human exception handling network that deals with the problem at great expense. But the cost to the development organization may be nothing more than reproducing and fixing the defect – expensive to be sure, but nothing compared with the cost of throwing an exception to the “flesh and bone” catch clause.

Worse, defects can damage the reputation of the product. Customers lose trust. Risks are introduced to the business that can go undetected for months or even years. The problems may be hard to stamp out completely, leading to strained customer relationships.

Part of the problem is that defects aren’t visible when software is delivered. Business leaders need to know the answers to two critical questions before the product is released:

• “Are there safety-critical software defects shipping in my products?”
• “Are there safety-critical software defects in my supplier’s products?”

This is one of the reasons we’ve launched the Coverity Integrity Report. It provides answers to those who care about the downstream costs of buggy software and the risks to the business that they can cause. The report shines a spotlight on the defects in software, how software compares with industry averages from open source code, and what the implications the defects might have on the application or system. In short, bugs can no longer hide.

Once the spotlight is put on bugs, will they matter? We’re going to find out.

These both demonstrate a lack of understanding with the automotive software assembly line. Likewise, this CNET article cites a vehicle tester at Edmunds.com who equates auto software to simple calculators. Again, not an engineer or programmer with a clear understanding of what goes on under the hood.

Two things stand out here:

First, auto software is not a standalone package. It is comprised of up to 100 million lines of code that come from multiple suppliers. In the Toyota case, it’s not clear whether the faulty brake software in the Prius was coded by Toyota engineers or by a supplier down the chain. And even if each individual code base was 100% bug free, there’s no guarantee that when the different pieces are glued together into a vehicle, there won’t be ‘glitches.’ To borrow from the Edmunds.com tester, it’s more accurate to say that auto software is more like different calculators duct-taped together.

Secondly, the Software Freedom Law Center’s call for Toyota to open source its code isn’t a realistic step. By its name, I get that the center promotes free software, and as such, are proponents of Linus’ Law that with enough eyeballs, code is better and more secure. Logically, this makes sense. But just how many eyeballs are enough? At a rough estimate, 50 lines of code are equal to one printed page. That means 100 million lines of code runs to about 2,000,000 pages.

The ingenuity and excellence of open source software development has created a wealth of excellent software. But as detailed in the 2009 Scan Open Source Report, there is a lot more to gain by using static analysis to improve the quality, security and integrity of software. Shawn Herman, a Microsoft programmer, wrote a very thoughtful piece on this subject looking at where human code review (eyeballs) stops and where automated analysis (static analysis and dynamic analysis) picks up. He explains that “a static analysis tool looks at source code the same way a compiler does, and so it has much more knowledge of what will happen than a human does.”

Instead of just open sourcing the software, the real effort needs to focus on Deming-like quality control for the automotive software assembly line. At a minimum, we believe that automated analysis needs to be part of this new software assembly line, preferably as early in the development process as possible. Right now vehicles are rolling consumer electronics devices but we expect that they will go through the same production maturity that we’ve seen in avionics and will be developed under mandated safety quality procedures.

Part of the answer is many companies have already switched. But many more have stuck with C/C++ (see the Tiobe data on language popularity for one measure). A few of the many reasons from observing our customers are:

• Legacy code. Consider the ton of code written in C/C++ out there. Coverity Scan analyzes over 300 of the most popular open source projects written in C/C++, including the Linux kernel and Firefox browser. There are about 60 million lines of code (MLOC) in these projects combined. In the commercial world, some of our larger customers have over 30 MLOC in a single project. To give you some perspective, 1 MLOC amounts to a stack of printed paper about 6 feet high. There would need to be a really, really good reason to consider rewriting anything of that magnitude.
• Risk aversion. Code “hardens” as it ages and enters repeated production use. Large code bases are tested over a period of years, if not decades. The reality is that all production software, no matter what language it’s written in, needs to be tested for functionality, stability, and performance reasons. Using a great language might help with stability to some degree, but rarely functionality or performance. Rewriting a (sub)system reaps certain benefits, but also introduces risks to existing functionality, stability, and performance. Sometimes, the risk-reward trade-off is worthwhile, but often it is not.
• Embedded software. Think mobile phones, cable set-top boxes, internet routers, wireless base stations, firewalls, network-attached storage systems, weapons control systems, and medical devices. They aren’t running on commodity PC hardware, processors, and operating systems. The hardware is often chosen for power-performance and cost reasons, and not infrequently there are large chunks of kernel-mode code. Sometimes these systems start from the Linux or FreeBSD kernel, or perhaps a commercial offering like Wind River VxWorks. Specialized hardware devices are common in this world, and low-level direct memory access is frequently necessary. Much of this software is written in C/C++.
• Performance and control. C/C++ compilers generally produce very efficient object code. Still, the choice of algorithm, and careful tuning and performance evaluation can trump language choice. However, in my experience, C/C++ provides a degree of flexibility and control that can sometimes help in optimizing the performance of a system in a way that many other languages don’t allow. Of course, the same flexibility and control can lead to bugs that are hard to diagnose.

When I was a researcher back at Stanford, my favorite programming language was O’Caml. I felt wonderfully productive writing code in a language that had type safety, type inference, pattern matching, and purely functional data structures (still a novelty at the time). We initially started off writing most of our core functionality in O’Caml, but eventually, had to abandon that implementation in favor of one written in C++. It was just too hard to hire programmers who knew O’Caml, and we also found that certain features that helped eliminate defects also reduced our control of the program’s performance, scalability behavior, and flexibility. Writing in C++ was often more verbose and less elegant, but it was possible to get to the result needed for the business.

At the end of the day, the features of a programming language are only one aspect of what makes it suitable or not for a particular job.