Weblogic Authors: Yeshim Deniz, Elizabeth White, Michael Meiner, Michael Bushong, Avi Rosenthal

Related Topics: @CloudExpo

@CloudExpo: Blog Post

Logs for Better Clouds - Part 7: Log Integrity

Not All Log Management Solutions Created Equal

Not all Log Management solutions are created equal... Trusting your logs.

Log Integrity is at the core of using logs for such purpose as building Trust, providing non-repudiation and indisputable proof in business relationships between Customers and Providers, but also to provide for evidence admissible in a court of law. We saw that not all Log Management solutions are created equal, and we saw some high-level requirements in terms of log collection and log reporting. We need a solution that is simple to deploy - we want an enabler, not a disabler - and a solution that allows a very rich set of APIs to accommodate for very different reporting on all kinds of metrics.

There are 2 other Critical Success Factors that need to be part of the "Not all Log Management Solutions are Created Equal" equation, and these are Log Integrity and Provider Reversibility. Today, more on Log Integrity.

Log Integrity
Logs can be used to foster Trust by providing non-repudiation and indisputable proof... provided that we can Trust the logs, in other words if we can guarantee their integrity. This is important not only for Trust for business relationship between a customer and its provider, but also in case of security breach as logs become prime evidence and need to be admissible in a court of law.  Imagine seeing malicious behavior but not being able to use your logs as evidence because you cannot guarantee that they have not been tampered with. It's like if you knew a crime was committed and you even have a picture of it, but this evidence is thrown away because you cannot prove that it was not photoshop'ed. Too bad...

We need 3 different proofs of integrity are demonstrated;

  1. Proof of integrity of each log - demonstrate that no log has been altered.
  2. Proof of integrity of the log sequence - demonstrate that no log has been added and no log has been deleted.
  3. Proof of integrity of the report - demonstrate that the report is complete and that all logs are reported on.

Once all of these are provided, there is Explicit Trust in raw logs. There are many ways of providing Log Integrity and Log Sequence Integrity, let's have a look at one of the easiest ways, to create a digitally signed - or at least a one-way hashed - chained file of logs.

In the following diagram Figure 7, we see how this can provide for log integrity and log sequence integrity.

Figure 7: Proof of log integrity through log block chaining and signing

In the following diagram Figure 8, we see that any modification of a log or any modification of the log sequence will be immediately detected and that we’ll be able to claim loss of integrity in logs. Without getting into implementation considerations, there are obvious tradeoffs on the size of each log block. The longer the block, the easier the management and the better the performance; the shorter the block, the fewer logs we have to throw away if we were to detect loss of integrity.

Figure 8: Loss of integrity detected

We can now trust the information contained in the raw logs as being genuine, and we can trust the information contained in the reports as being non-tainted. Report completeness needs to be guaranteed by the tool, in other words, there needs to be built-in mechanisms that insure that all logs that need to be part of a report are included in the report generation and computation. This is an inherent function of the tool. We can never prevent accidental or malicious modification of a log, but we can detect modifications with a simple yet powerful way, Log Block Chaining and Signing. This will insure that the logs that we work from are genuine, have not been modified, and represent a clean source of data on top on which we can build non-repudiation and proof of claim, we can claim Trust.

More Stories By Gorka Sadowski

Gorka is a natural born entrepreneur with a deep understanding of Technology, IT Security and how these create value in the Marketplace. He is today offering innovative European startups the opportunity to benefit from the Silicon Valley ecosystem accelerators. Gorka spent the last 20 years initiating, building and growing businesses that provide technology solutions to the Industry. From General Manager Spain, Italy and Portugal for LogLogic, defining Next Generation Log Management and Security Forensics, to Director Unisys France, bringing Cloud Security service offerings to the market, from Director of Emerging Technologies at NetScreen, defining Next Generation Firewall, to Director of Performance Engineering at INS, removing WAN and Internet bottlenecks, Gorka has always been involved in innovative Technology and IT Security solutions, creating successful Business Units within established Groups and helping launch breakthrough startups such as KOLA Kids OnLine America, a social network for safe computing for children, SourceFire, a leading network security solution provider, or Ibixis, a boutique European business accelerator.

IoT & Smart Cities Stories
DXWorldEXPO LLC announced today that ICOHOLDER named "Media Sponsor" of Miami Blockchain Event by FinTechEXPO. ICOHOLDER gives detailed information and help the community to invest in the trusty projects. Miami Blockchain Event by FinTechEXPO has opened its Call for Papers. The two-day event will present 20 top Blockchain experts. All speaking inquiries which covers the following information can be submitted by email to [email protected] Miami Blockchain Event by FinTechEXPOalso offers sp...
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world. The widespread success of cloud computing is driving the DevOps revolution in enterprise IT. Now as never before, development teams must communicate and collaborate in a dynamic, 24/7/365 environment. There is no time t...
Headquartered in Plainsboro, NJ, Synametrics Technologies has provided IT professionals and computer systems developers since 1997. Based on the success of their initial product offerings (WinSQL and DeltaCopy), the company continues to create and hone innovative products that help its customers get more from their computer applications, databases and infrastructure. To date, over one million users around the world have chosen Synametrics solutions to help power their accelerated business or per...
When talking IoT we often focus on the devices, the sensors, the hardware itself. The new smart appliances, the new smart or self-driving cars (which are amalgamations of many ‘things'). When we are looking at the world of IoT, we should take a step back, look at the big picture. What value are these devices providing. IoT is not about the devices, its about the data consumed and generated. The devices are tools, mechanisms, conduits. This paper discusses the considerations when dealing with the...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
Machine learning has taken residence at our cities' cores and now we can finally have "smart cities." Cities are a collection of buildings made to provide the structure and safety necessary for people to function, create and survive. Buildings are a pool of ever-changing performance data from large automated systems such as heating and cooling to the people that live and work within them. Through machine learning, buildings can optimize performance, reduce costs, and improve occupant comfort by ...
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
Bill Schmarzo, Tech Chair of "Big Data | Analytics" of upcoming CloudEXPO | DXWorldEXPO New York (November 12-13, 2018, New York City) today announced the outline and schedule of the track. "The track has been designed in experience/degree order," said Schmarzo. "So, that folks who attend the entire track can leave the conference with some of the skills necessary to get their work done when they get back to their offices. It actually ties back to some work that I'm doing at the University of San...
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.