Weblogic Authors: Yeshim Deniz, Elizabeth White, Michael Meiner, Michael Bushong, Avi Rosenthal

Related Topics: Weblogic

Weblogic: Article

Recovering from an Invalid System Password

Recovering from an Invalid System Password

You might not want to admit it, but have you ever lost or forgotten the password for the system user within WebLogic Server (WLS) 6.1? Or worse yet, accidentally deleted the fileRealm. properties or SerializedSystemIni.dat files?

If so, you know the consequence - failure to produce a valid system password at server startup prevents the server instance from booting. This article shows you how to recover from this type of situation, keeping production downtime to a minimum.

Losing the System Password
As most administrators know, system password management within the production environment is critical. Root passwords are required to manage a wide array of operational resources, ranging from operating systems to network routers. A production environment is directly impacted when a key system password is lost, rendering the affected resource inaccessible until an administrator recovers or replaces the password.

With WLS, losing the system password within the default file security realm will prevent a server instance from starting up, blocking its boot process until the system password is reset or recovered. The problem is further complicated by the fact that WLS password management is promoted by the browser-based administration console, a valuable interface constrained by the limitation that the server must be up and running in order for it to be accessible. What options are available when the system password is lost but can't be reset from the console? The answer lies in a command-line feature hidden within the file realm itself.

Elements of the File Realm

Security realms play an important role within WLS 6.1; they define the set of users, passwords, groups, and access control lists used by a domain for its base security information. While various types of security realms exist, WLS by default uses the file realm unless configured otherwise.

For a given domain utilizing the file realm, two crucial files work together and constitute the underlying repository of the security realm. The first, a text file, named fileRealm.properties, persistently defines the actual set of users, hashed passwords, groups, and access control lists of the realm. The second, a binary file named SerializedSystemIni.dat, defines the seed input used by the file realm when a cleartext password is hashed in its SHA-based format. Both files exist within the root directory of the given domain (e.g., ./config/mydomain).

Since the SerializedSystemIni.dat file encapsulates a time-variant encrypted key created when a domain is first generated, no two SerializedSystemIni.dat files are created alike for a specific server installation. Therefore, when a cleartext password is hashed for a certain user and stored within the fileRealm.properties file, an explicit relationship is immediately established between the fileRealm.properties repository and its input SerializedSystemIni.dat file. The password stored in the repository and hashed with the SerializedSystemIni.dat input can be successfully compared only to its cleartext equivalent hashed with the same SerializedSystemIni.dat. This element is critical to the validation process of password comparison, where a cleartext password is compared to a stored password at the hash level, since it's impossible to directly reverse the hashed password back out to its original cleartext format.

An Invalid Password - More Than Just Forgetful
With WLS 6.1, the system password can be invalidated in a few ways. Aside from the obvious cases where the password is forgotten or misplaced (considered a lost password), the system password for the file realm of a given domain also becomes invalid when one of the following situations occurs:

  • Errant password modification: The fileRealm.properties file has been modified in such a way that an errant change is introduced to the hashed password value for the system user (intentional or otherwise).
  • Mismatched seed and repository: The SerializedSystemIni.dat file has been modified, replaced, or removed entirely, introducing a change that breaks the relationship between a given salt file and the set of passwords stored within a specific fileRealm.properties file.

An errant modification to a password is equivalent to password corruption. It usually results from one of two possibilities, differing only by the intent - an accidental change by an administrator who inadvertently modifies a password during an edit session, or a planned attack by a malicious user who intentionally corrupts a password. In both scenarios, the original hashed value of the password is changed at the character level so that a new hash is created, making it impossible to determine what the original cleartext of the new hash is and rendering the password invalid.

In the scenario involving a mismatched seed and repository, the relationship between a given salt file and the set of passwords stored within a specific fileRealm.properties file is broken as a result of the SerializedSystemIni.dat seed file having been modified, removed, or replaced entirely by another file (seed or otherwise). When the seed file becomes invalid with respect to its relationship in this manner, password comparison at the hash level will fail, since passwords stored within a given fileRealm.properties file and initially hashed with one salt are now compared to passwords hashed with another salt.

Available Recovery Options
When the system password is invalidated by one of the above possibilities, the recovery options are limited. If the password has been lost or corrupted, an administrator can restore the password from a backup copy of the fileRealm.properties file, if one exists. If a seed file is invalidated and its relationship to the set of passwords within a given repository has been broken, an administrator can restore the salt from a backup copy of the SerializedSystemIni.dat file - again, if the backup exists.

What can be done, however, when backup copies of the seed and repository files don't exist for the affected domain? You can copy the hashed passwords and appropriate files from the file realm of another domain (where the values of those passwords are known) when another domain exists, or create a new domain by installing a clean instance of WLS and copying from that. Of course, this has problems in its own right, as it depends upon the existence of another domain or the creation of a new domain through installation, and involves a fair amount of manual interaction. Besides, you might not want to risk compromising a production environment with passwords from another configuration.

When backup restoration isn't available and domain copies are not possible or desired, the best recovery option is to reset the password from the command line using the features of a hidden utility within the file realm itself. Mastering this utility will ensure proper recovery from disaster and enable a production environment to operate with little interruption. The rest of this article discusses in detail this utility - the FileRealm class.

The FileRealm Class
Every type of security realm within WLS 6.1 is ultimately managed and represented by a pure Java class. Such classes are either provided with the product out of the box or are externally developed to meet specific security needs. The default file realm is no exception, and is governed by the internal weblogic.security.acl.internal.FileRealm class that ships with WLS (packaged as part of the core weblogic.jar file). The primary responsibilities of this class are to manage the users, passwords, groups, and access control lists of the file-based security realm and provide authentication and authorization services for clients, using the underlying fileRealm.properties file as its persistent security repository and master record of data.

By all accounts, the FileRealm class should offer nothing more than a set of interface methods that other internal components of WLS can publicly use when authenticating and authorizing client access within the security realm - and it does. Interestingly enough, however, the FileRealm class also exposes a main method, allowing it to be invoked from the command line as a normal executable program. As a result, the internal class is surprisingly dual-purposed, serving primarily as a supporting library class to other elements within WLS, yet unexpectedly promoting itself as a command-line utility at the same time.

When executed from the command line, the FileRealm class provides a mechanism by which a cleartext password for a given user can be hashed with a specified input seed file, producing an encrypted equivalent output that can be stored within the fileRealm.properties repository and subsequently referenced by the file realm as the actual password for the given user. In this way, the class effectively gives administrators the ability to set passwords from the command line for any user within the file realm, free of the limitations imposed by the browser-based administration console and the password management system it offers. The potential value offered by this internal class is only fully realized in the most disastrous of situations, where the system password has been lost or invalidated and must be reset from the command line when the server consequently fails to start.

Resetting the System Password
Utilizing the command-line feature of the FileRealm class to set or change a password for a user within the file-based security realm is relatively straightforward. You must first identify the set of users that require password changes - the FileRealm class allows you to consolidate all password changes for multiple users at once, so executing the program in multiple iterations to handle a batch of users on an individual basis isn't required. However, in the example below we'll change the password for only one user (the system user).

When the set of users has been identified, a properties file must be created, and the users need to be defined within this file. Comments are allowed within the file (preceded by the # sign), and for each user within the set, a corresponding entry must be defined on its own new line within the file using the following format:


An example follows:

# define the user system to have the
cleartext password weblogic

This file provides the input list of users for the FileRealm class to process and associate each user with its original cleartext password. For this reason, the properties file can also be referred to as the input definition file. This file can be located anywhere in the local file system, and must end with the .src extension. In the example used below, we'll define the input definition file as user.properties.src.

When the input definition file has been created, the location of SerializedSystemIni.dat must be determined before the FileRealm class is executed from the command line. Recall that SerializedSystemIni.dat provides an input seed (or salt) to the hashing phase of the encryption process, and that a password hashed with a specific salt can be successfully compared only to another password hashed by the same exact salt. Therefore, the SerializedSystemIni.dat file, localized at the domain level and present within the config/ directory, has an explicit relationship to the passwords it hashes for the file realm of a given domain. Since WLS can't maintain a file realm in which the set of hashed passwords present have been seeded by different SerializedSystemIni.dat files, you should never mix passwords hashed by different salt files within the same file realm.

Depending upon the situation and the specifics of your problem, the FileRealm class can use an existing SerializedSystemIni.dat file for its input seed or create a new one if none exists. Referencing an existing SerializedSystemIni.dat is beneficial in the scenario in which the system password for a given domain has been lost but the domain-level salt file is still present and valid. This forces the FileRealm class to create a new password with the same seed used to hash all other passwords within the domain. Creating a new SerializedSystemIni.dat is useful when the salt file has been deleted or corrupted and the relationship to the passwords hashed by the original salt has been broken, making them invalid. Note that when a new SerializedSystemIni.dat file is created by the FileRealm class and used to hash new passwords for a domain, all other passwords within that same domain must be reset as well, since the seed for the hash has changed.

Executing the FileRealm class is simple and straightforward. Set your system classpath to include the weblogic.jar file from the lib directory of the WLS installation and invoke the utility from the command line as follows:

java weblogic.security.acl.internal.FileRealm \ <path_to_output_file> \ <path_to_salt_file>

Upon processing the input definition file, the FileRealm class generates its resultant set of users and their hashed passwords into the output file defined by the first parameter. This file is identical to the input definition file in every way except for the format of the passwords - the passwords in the output file are hashed, while those from the input are cleartext. Of course, this is the desired result. The path to the output file given by the first parameter should be identical to that of the input definition file, with the .src extension dropped for the output file. For example, if you had created the input definition file from above at /tmp/user.properties.src, you'd now need to define the output file at /tmp/user.properties. This accommodates the way the FileRealm class internally handles the location of the input definition file - it concludes its location by using the same path and name of the output file, yet it assumes the input file has the .src extension added to it.

The second parameter defines the location of the SerializedSystemIni.dat file. If the relative or absolute path specified references an existing salt file, then that seed is used as the input. If not, a new seed is created at the path specified with the given filename. Note that while it's possible to reference or create a new salt file with a name other than SerializedSystemIni.dat and outside of a given domain directory, WLS will recognize at runtime only seeds named SerializedSystemIni.dat and placed within their appropriate domain directories.

Upon execution, the FileRealm class will generate its hashed password outputs into the file defined by the first input parameter. When the command-line utility has completed, the invalid passwords found within the fileRealm.properties repository of the affected file realm should be overwritten with the new passwords stored within the output file. This last step should be done manually using your favorite text editor and a few select copy/paste operations. Once the changes have been made to fileRealm.properties and the file has been saved, your invalid passwords are valid once again, and must be verified. Start the server and provide the password recently set for the system user. If the password recovery was successful, the server will properly start and WLS will operate as expected. When you're finished, remove the input definition file and its output equivalent from the filesystem to avoid compromising your newly set passwords.

The browser-based administration console, an excellent utility constrained by the requirement of a running server, is the supported mechanism for password management within WLS 6.1. When the system password has been lost or invalidated and WLS won't start, however, the console becomes inaccessible and the only reliable option for resetting a file realm password is to do so from the command line. Utilizing the hidden features of the FileRealm class, you can manage passwords from the command line and recover from an otherwise disastrous situation. Additionally, while the hidden features of the FileRealm class discussed within this article relate to WLS 6.1, they are fully accessible and available under WLS 7.0, and should be used for the same purposes when running in the 6.1-compatible security mode.

More Stories By Steve Mueller

Steve Mueller is a principal
consultant for BEA Systems, where he specializes in the design,
development, and administration of enterprise systems running on WebLogic Server.

Comments (0)

Share your thoughts on this story.

Add your comment
You must be signed in to add a comment. Sign-in | Register

In accordance with our Comment Policy, we encourage comments that are on topic, relevant and to-the-point. We will remove comments that include profanity, personal attacks, racial slurs, threats of violence, or other inappropriate material that violates our Terms and Conditions, and will block users who make repeated violations. We ask all readers to expect diversity of opinion and to treat one another with dignity and respect.

@ThingsExpo Stories
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of bus...
Cloud Expo | DXWorld Expo have announced the conference tracks for Cloud Expo 2018. Cloud Expo will be held June 5-7, 2018, at the Javits Center in New York City, and November 6-8, 2018, at the Santa Clara Convention Center, Santa Clara, CA. Digital Transformation (DX) is a major focus with the introduction of DX Expo within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive ov...
DXWordEXPO New York 2018, colocated with CloudEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
DXWorldEXPO LLC announced today that ICOHOLDER named "Media Sponsor" of Miami Blockchain Event by FinTechEXPO. ICOHOLDER give you detailed information and help the community to invest in the trusty projects. Miami Blockchain Event by FinTechEXPO has opened its Call for Papers. The two-day event will present 20 top Blockchain experts. All speaking inquiries which covers the following information can be submitted by email to [email protected] Miami Blockchain Event by FinTechEXPO also offers s...
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
Digital Transformation and Disruption, Amazon Style - What You Can Learn. Chris Kocher is a co-founder of Grey Heron, a management and strategic marketing consulting firm. He has 25+ years in both strategic and hands-on operating experience helping executives and investors build revenues and shareholder value. He has consulted with over 130 companies on innovating with new business models, product strategies and monetization. Chris has held management positions at HP and Symantec in addition to ...
Cloud-enabled transformation has evolved from cost saving measure to business innovation strategy -- one that combines the cloud with cognitive capabilities to drive market disruption. Learn how you can achieve the insight and agility you need to gain a competitive advantage. Industry-acclaimed CTO and cloud expert, Shankar Kalyana presents. Only the most exceptional IBMers are appointed with the rare distinction of IBM Fellow, the highest technical honor in the company. Shankar has also receive...
Enterprises have taken advantage of IoT to achieve important revenue and cost advantages. What is less apparent is how incumbent enterprises operating at scale have, following success with IoT, built analytic, operations management and software development capabilities - ranging from autonomous vehicles to manageable robotics installations. They have embraced these capabilities as if they were Silicon Valley startups.
Poor data quality and analytics drive down business value. In fact, Gartner estimated that the average financial impact of poor data quality on organizations is $9.7 million per year. But bad data is much more than a cost center. By eroding trust in information, analytics and the business decisions based on these, it is a serious impediment to digital transformation.
The standardization of container runtimes and images has sparked the creation of an almost overwhelming number of new open source projects that build on and otherwise work with these specifications. Of course, there's Kubernetes, which orchestrates and manages collections of containers. It was one of the first and best-known examples of projects that make containers truly useful for production use. However, more recently, the container ecosystem has truly exploded. A service mesh like Istio addr...
Predicting the future has never been more challenging - not because of the lack of data but because of the flood of ungoverned and risk laden information. Microsoft states that 2.5 exabytes of data are created every day. Expectations and reliance on data are being pushed to the limits, as demands around hybrid options continue to grow.
As IoT continues to increase momentum, so does the associated risk. Secure Device Lifecycle Management (DLM) is ranked as one of the most important technology areas of IoT. Driving this trend is the realization that secure support for IoT devices provides companies the ability to deliver high-quality, reliable, secure offerings faster, create new revenue streams, and reduce support costs, all while building a competitive advantage in their markets. In this session, we will use customer use cases...
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Digital Transformation: Preparing Cloud & IoT Security for the Age of Artificial Intelligence. As automation and artificial intelligence (AI) power solution development and delivery, many businesses need to build backend cloud capabilities. Well-poised organizations, marketing smart devices with AI and BlockChain capabilities prepare to refine compliance and regulatory capabilities in 2018. Volumes of health, financial, technical and privacy data, along with tightening compliance requirements by...
With 10 simultaneous tracks, keynotes, general sessions and targeted breakout classes, @CloudEXPO and DXWorldEXPO are two of the most important technology events of the year. Since its launch over eight years ago, @CloudEXPO and DXWorldEXPO have presented a rock star faculty as well as showcased hundreds of sponsors and exhibitors! In this blog post, we provide 7 tips on how, as part of our world-class faculty, you can deliver one of the most popular sessions at our events. But before reading...
The best way to leverage your Cloud Expo presence as a sponsor and exhibitor is to plan your news announcements around our events. The press covering Cloud Expo and @ThingsExpo will have access to these releases and will amplify your news announcements. More than two dozen Cloud companies either set deals at our shows or have announced their mergers and acquisitions at Cloud Expo. Product announcements during our show provide your company with the most reach through our targeted audiences.
The IoT Will Grow: In what might be the most obvious prediction of the decade, the IoT will continue to expand next year, with more and more devices coming online every single day. What isn’t so obvious about this prediction: where that growth will occur. The retail, healthcare, and industrial/supply chain industries will likely see the greatest growth. Forrester Research has predicted the IoT will become “the backbone” of customer value as it continues to grow. It is no surprise that retail is ...
Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settlement products to hedge funds and investment banks. After, he co-founded a revenue cycle management company where he learned about Bitcoin and eventually Ethereal. Andrew's role at ConsenSys Enterprise is a mul...
DXWorldEXPO LLC announced today that "Miami Blockchain Event by FinTechEXPO" has announced that its Call for Papers is now open. The two-day event will present 20 top Blockchain experts. All speaking inquiries which covers the following information can be submitted by email to [email protected] Financial enterprises in New York City, London, Singapore, and other world financial capitals are embracing a new generation of smart, automated FinTech that eliminates many cumbersome, slow, and expe...