Log4j Vulnerability Information

[Jonathan Braley]

We are starting this thread and making it available to the public, as a community service, due to the severity of this active exploit. Please post all updates and new information to this thread.​

---

Pinning this important resource to the top of this thread for maximum visibility:

Sharing this important list of vulnerable and safe software. Will pin this to the top of this thread as well. It's also embedded in the resource list further up/down (depending where you are reading this in the list) in this thread.

log4shell/software at main · NCSC-NL/log4shell

Operational information regarding the log4shell vulnerabilities in the Log4j logging library. - NCSC-NL/log4shell

github.com

---

--

A Zero-day Exploit for Log4j Java Library Could Have a Tsunami Impact on IT Giants

Summary:
“Experts publicly disclose Proof-of-concept exploits for a critical remote code execution zero-day vulnerability, tracked a CVE-2021-44228 (aka Log4Shell), in the Apache Log4j Java-based logging library. The Chinese security researcher p0rz9 who publicly disclosed the PoC exploit code revealed that the CVE-2021-44228 can only be exploited if the log4j2.formatMsgNoLookups option is set to false” (Security Affairs, 2021).

The Log4j is widely used by both enterprise apps and cloud services, including Apple iCloud and Steam.

Analyst Comments:
The vulnerability was assigned CVE-2021-44228, it allows an unauthenticated attacker to execute arbitrary code on a vulnerable system leading to complete system takeover.

Most alarming, the vulnerability does not require any special configurations which is why it received a CVSS score of 10/10. Apache Struts2, Apache Solr, Apache Druid, Apache Flink are all affected by this vulnerability. Open-source projects like ElasticSearch, Elastic Logstash, Redis, and the NSA’s Ghidra also use the library.

“IT giants like Apple, Amazon, Twitter, Cloudflare, Steam, Tencent, Baidu, and NetEase are running servers potentially affected by the issue. Researchers from Bad Packets are already observing mass scanning activity for this vulnerability. Lunasec, who tracked this vulnerability as LogJam, confirmed the wide impact of this issue” (Security Affairs, 2021).

Mitigation:
Apache addressed the issue with the release of a Log4j release candidate version (2.15.0-rc1), but security researchers already discovered a bypass and are urging impacted organizations to update to the latest RC build log4j-2.15.0-rc2.

Source:
https://securityaffairs.co/wordpress/125480/hacking/log4j-java-library-zeroday.html

--

Greynoise is sharing IPs it identified as exploiting this vuln - you can find details here:

https://twitter.com/x/status/1469326260803416073

--

https://twitter.com/x/status/1469248250670727169

 

[Jonathan Braley]

TruSTAR IoCs from GreyNoise: https://station.trustar.co/constellation/reports/f1c259c8-e013-4709-88b3-46ff9965b9b3

 

[Jonathan Braley]

You can use these commands and rules to search for exploitation attempts against log4j RCE vulnerability CVE-2021-44228

--

Log4j RCE CVE-2021-44228 Exploitation Detection

Log4j RCE CVE-2021-44228 Exploitation Detection. GitHub Gist: instantly share code, notes, and snippets.

gist.github.com

Attack Surface: https://github.com/YfryTchsGD/Log4jAttackSurface

 

[Nick Hansen]

This is a big one, thanks for the extra details and information!

 

[Jonathan Braley]

GreyNoise IoCs TXT file. If you are unfamiliar, GreyNoise sets up vulnerable honeypots and collects IPs of active exploitation. So these are all likely known bad.

 

[Jonathan Braley]

Update - 12-11-2021 - https://www.cisa.gov/news/2021/12/11/statement-cisa-director-easterly-log4j-vulnerability

--

WASHINGTON – Cybersecurity and Infrastructure Security Agency (CISA) Director Jen Easterly released the following statement today on the “log4j” vulnerability:

“CISA is working closely with our public and private sector partners to proactively address a critical vulnerability affecting products containing the log4j software library. This vulnerability, which is being widely exploited by a growing set of threat actors, presents an urgent challenge to network defenders given its broad use. End users will be reliant on their vendors, and the vendor community must immediately identify, mitigate, and patch the wide array of products using this software. Vendors should also be communicating with their customers to ensure end users know that their product contains this vulnerability and should prioritize software updates.

“We are taking urgent action to drive mitigation of this vulnerability and detect any associated threat activity. We have added this vulnerability to our catalog of known exploited vulnerabilities, which compels federal civilian agencies -- and signals to non-federal partners -- to urgently patch or remediate this vulnerability. We are proactively reaching out to entities whose networks may be vulnerable and are leveraging our scanning and intrusion detection tools to help government and industry partners identify exposure to or exploitation of the vulnerability.

“The Joint Cyber Defense Collaborative is designed to manage this kind of risk. We have established a JCDC senior leadership group to coordinate collective action and ensure shared visibility into both the prevalence of this vulnerability and threat activity. By bringing together key government and private sector partners via the JCDC, including our partners at the FBI and NSA, we will ensure that our country’s strongest capabilities are brought to bear in an integrated manner against this risk. To ensure the broadest possible dissemination of key information, we are also convening a national call with critical infrastructure stakeholders on Monday afternoon where CISA’s experts provide further insight and address questions.

“We continue to urge all organizations to review the latest CISA current activity alert and upgrade to log4j version 2.15.0, or apply their appropriate vendor recommended mitigations immediately.

“To be clear, this vulnerability poses a severe risk. We will only minimize potential impacts through collaborative efforts between government and the private sector. We urge all organizations to join us in this essential effort and take action.”

CISA recommends asset owners take three additional, immediate steps regarding this vulnerability:

1. Enumerate any external facing devices that have log4j installed.
2. Make sure that your security operations center is actioning every single alert on the devices that fall into the category above.
3. Install a web application firewall (WAF) with rules that automatically update so that your SOC is able to concentrate on fewer alerts.

This effort also underscores the urgency of building software securely from the start and more widespread use of Software Bill of Materials (SBOM), both of which were directed by President Biden in his Executive Order issued in May 2021. A SBOM would provide end users will the transparency they require to know if their products rely on vulnerable software libraries.

 

[Raffi Jamgotchian]

Update - 12-11-2021 - https://www.cisa.gov/news/2021/12/11/statement-cisa-director-easterly-log4j-vulnerability

--

WASHINGTON – Cybersecurity and Infrastructure Security Agency (CISA) Director Jen Easterly released the following statement today on the “log4j” vulnerability:

“CISA is working closely with our public and private sector partners to proactively address a critical vulnerability affecting products containing the log4j software library. This vulnerability, which is being widely exploited by a growing set of threat actors, presents an urgent challenge to network defenders given its broad use. End users will be reliant on their vendors, and the vendor community must immediately identify, mitigate, and patch the wide array of products using this software. Vendors should also be communicating with their customers to ensure end users know that their product contains this vulnerability and should prioritize software updates.

“We are taking urgent action to drive mitigation of this vulnerability and detect any associated threat activity. We have added this vulnerability to our catalog of known exploited vulnerabilities, which compels federal civilian agencies -- and signals to non-federal partners -- to urgently patch or remediate this vulnerability. We are proactively reaching out to entities whose networks may be vulnerable and are leveraging our scanning and intrusion detection tools to help government and industry partners identify exposure to or exploitation of the vulnerability.

“The Joint Cyber Defense Collaborative is designed to manage this kind of risk. We have established a JCDC senior leadership group to coordinate collective action and ensure shared visibility into both the prevalence of this vulnerability and threat activity. By bringing together key government and private sector partners via the JCDC, including our partners at the FBI and NSA, we will ensure that our country’s strongest capabilities are brought to bear in an integrated manner against this risk. To ensure the broadest possible dissemination of key information, we are also convening a national call with critical infrastructure stakeholders on Monday afternoon where CISA’s experts provide further insight and address questions.

“We continue to urge all organizations to review the latest CISA current activity alert and upgrade to log4j version 2.15.0, or apply their appropriate vendor recommended mitigations immediately.

“To be clear, this vulnerability poses a severe risk. We will only minimize potential impacts through collaborative efforts between government and the private sector. We urge all organizations to join us in this essential effort and take action.”

CISA recommends asset owners take three additional, immediate steps regarding this vulnerability:

1. Enumerate any external facing devices that have log4j installed.
2. Make sure that your security operations center is actioning every single alert on the devices that fall into the category above.
3. Install a web application firewall (WAF) with rules that automatically update so that your SOC is able to concentrate on fewer alerts.

This effort also underscores the urgency of building software securely from the start and more widespread use of Software Bill of Materials (SBOM), both of which were directed by President Biden in his Executive Order issued in May 2021. A SBOM would provide end users will the transparency they require to know if their products rely on vulnerable software libraries.

Clearly an SBOM would be useful to understand what systems you have that could be affected by a flaw in a library such as this. I wonder how many software vendors are willing to publicly acknowledge what is being used and do they even know what their developers are pulling in?

Huntress made a blog post on this flaw that also includes a way to test if a system has this flaw by passing a POC string that would go through the library on its way to whatever logging mechanism has this issue:

Critical RCE Vulnerability Updates (log4j - CVE-2021-44228) | Huntress

Read about how our team investigated CVE-2021-44228, a critical vulnerability that was affecting a Java logging package.

www.huntress.com

Tom Lawrence also put together a good "lay-IT" description of the flow.

 

[Jonathan Braley]

FYSA

--

Critical Infrastructure Colleagues and Partners,

***For widest distribution amongst CISA partners--- Not for media representatives***

The Cybersecurity and Infrastructure Security Agency (CISA) invites you to participate on a call Monday, December 13, 2021, at 1:30 pm Eastern to discuss the Apache Log4j Vulnerability (CVE-2021-44228). Subject Matter Experts from CISA will provide details about the vulnerability and mitigation strategies.

CISA strongly urges users and administrators to review the Apache Log4j 2.15.0 Announcement and upgrade to Log4j 2.15.0 or apply the recommended mitigations immediately.

Date/Time: Monday, December 13, 2021 (1:30pm EST)

Participant Toll Free Dial in Number: 1- 800-857-9685 (passcode 2168780)

International Dial in Number: 1-773-799-3857 (passcode 2168780)


Respectfully,

Cybersecurity and Infrastructure Security Agency

 

[Jonathan Braley]

Clearly an SBOM would be useful to understand what systems you have that could be affected by a flaw in a library such as this. I wonder how many software vendors are willing to publicly acknowledge what is being used and do they even know what their developers are pulling in?

Huntress made a blog post on this flaw that also includes a way to test if a system has this flaw by passing a POC string that would go through the library on its way to whatever logging mechanism has this issue:

Critical RCE Vulnerability Updates (log4j - CVE-2021-44228) | Huntress

Read about how our team investigated CVE-2021-44228, a critical vulnerability that was affecting a Java logging package.

www.huntress.com

Tom Lawrence also put together a good "lay-IT" description of the flow.

Could not agree more! Software Build of Materials is a great idea. There has definitely been a push for it, for those unaware of this concept, you basically have an "ingredients list" similar to what food has, but for software. It would contain the various components of software so when a vulnerability like this comes out, you can quickly see where remediation is needed.

 

[Jonathan Braley]

log4j2 CVE-2021-44228 patched three class files - https://gitbox.apache.org/repos/asf?p=logging-log4j2.git;h=c77b3cb

Below are the inner class files (i.e. not jar) for each release to help locate vulnerable versions. Note: 2.0.15 was the patched version and is not vulnerable.


https://gist.github.com/olliencc/8be866ae94b6bee107e3755fd1e9bf0d

 

[Ian Thornton-Trump CD]

Greetings.

This is way way overhyped.

See step 3 - Block LDAP 636 outbound at your load balancer and/or perimeter firewall. You can easily detect this attack. Most CDN's are blocking it & most EDR vendors are killing it at the network layer and payload deployment attempts.

Phat_Hobbit

 

[MJ Shoer]

Here is the original threat report that was issued Friday, December 10, 2021 at 10:45 AM EST:

Severity: Medium TLP: Green A Zero-day Exploit for Log4j Java Library Could Have a Tsunami Impact on IT Giants​

Summary:
“Experts publicly disclose Proof-of-concept exploits for a critical remote code execution zero-day vulnerability, tracked a CVE-2021-44228 (aka Log4Shell), in the Apache Log4j Java-based logging library. The Chinese security researcher p0rz9 who publicly disclosed the PoC exploit code revealed that the CVE-2021-44228 can only be exploited if the log4j2.formatMsgNoLookups option is set to false” (Security Affairs, 2021).

The Log4j is widely used by both enterprise apps and cloud services, including Apple iCloud and Steam.

Analyst Comments:
The vulnerability was assigned CVE-2021-44228, it allows an unauthenticated attacker to execute arbitrary code on a vulnerable system leading to complete system takeover.

Most alarming, the vulnerability does not require any special configurations which is why it received a CVSS score of 10/10. Apache Struts2, Apache Solr, Apache Druid, Apache Flink are all affected by this vulnerability. Open-source projects like ElasticSearch, Elastic Logstash, Redis, and the NSA’s Ghidra also use the library.

“IT giants like Apple, Amazon, Twitter, Cloudflare, Steam, Tencent, Baidu, and NetEase are running servers potentially affected by the issue. Researchers from Bad Packets are already observing mass scanning activity for this vulnerability. Lunasec, who tracked this vulnerability as LogJam, confirmed the wide impact of this issue” (Security Affairs, 2021).

Mitigation:
Apache addressed the issue with the release of a Log4j release candidate version (2.15.0-rc1), but security researchers already discovered a bypass and urge impacted organizations to updating to the latest RC build log4j-2.15.0-rc2.

Source:
https://securityaffairs.co/wordpress/125480/hacking/log4j-java-library-zeroday.html

 

[MJ Shoer]

Greetings.

This is way way overhyped.

See step 3 - Block LDAP 636 outbound at your load balancer and/or perimeter firewall. You can easily detect this attack. Most CDN's are blocking it & most EDR vendors are killing it at the network layer and payload deployment attempts.

Phat_Hobbit





View attachment 749

Excellent information @Ian Thornton-Trump CD. Thank you for sharing this.

 

[Ian Thornton-Trump CD]

and say you can't even manage the above well Apache provided this:

Apache Logging Services

CVE-2021-44228: Apache Log4j2 JNDI features do not protect against attacker controlled LDAP and other JNDI related endpoints.

Severity: Critical

Base CVSS Score: 10.0 CVSS:3.0/AV:N/AC:L/PR:N/UI:N/S:C/C:H/I:H/A:H

Versions Affected: all versions from 2.0-beta9 to 2.14.1

Descripton: Apache Log4j2 <=2.14.1 JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled. From log4j 2.15.0, this behavior has been disabled by default.

Mitigation: In releases >=2.10, this behavior can be mitigated by setting either the system property log4j2.formatMsgNoLookups or the environment variable LOG4J_FORMAT_MSG_NO_LOOKUPS to true. For releases from 2.0-beta9 to 2.10.0, the mitigation is to remove the JndiLookup class from the classpath: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class.

So please dial it down to a reasonable level there ARE PLENTY of mitigations.

 

[MJ Shoer]

Some additional resources from CompTIA ISAO members and partners:

Log4Shell Hell: anatomy of an exploit outbreak

Log4Shell explained – how it works, why you need to know, and how to fix it

“Log4Shell” Java vulnerability – how to safeguard your servers

Apache Releases Log4j Version 2.15.0 to Address Critical RCE Vulnerability Under Exploitation

Apache Log4j Security Vulnerabilities

CISA Apache Log4j Vulnerability Guidance

Huntress Log4Shell Vulnerability Tester

SANS: Log4j: Getting ready for the long haul (CVE-2021-44228)

Reddit Resources:

reddit.com: search results - log4j

www.reddit.com

Github Resources (use at your own risk):

GitHub - fullhunt/log4j-scan: A fully automated, accurate, and extensive scanner for finding log4j RCE CVE-2021-44228

A fully automated, accurate, and extensive scanner for finding log4j RCE CVE-2021-44228 - GitHub - fullhunt/log4j-scan: A fully automated, accurate, and extensive scanner for finding log4j RCE CVE...

github.com

GitHub - Neo23x0/log4shell-detector: Detector for Log4Shell exploitation attempts

Detector for Log4Shell exploitation attempts. Contribute to Neo23x0/log4shell-detector development by creating an account on GitHub.

github.com

https://gist.github.com/SwitHak/b66db3a06c2955a9cb71a8718970c592

GitHub - ossf/scorecard: OpenSSF Scorecard - Security health metrics for Open Source

OpenSSF Scorecard - Security health metrics for Open Source - ossf/scorecard

github.com

log4shell/software at main · NCSC-NL/log4shell

Operational information regarding the log4shell vulnerabilities in the Log4j logging library. - NCSC-NL/log4shell

github.com

 

[Jason Slagle]

Greetings.

This is way way overhyped.

See step 3 - Block LDAP 636 outbound at your load balancer and/or perimeter firewall. You can easily detect this attack. Most CDN's are blocking it & most EDR vendors are killing it at the network layer and payload deployment attempts.

Phat_Hobbit





View attachment 749

Except that pretty much everyone is not running the LDAP server on 636 - it's almost always on a non-privileged port.

There are tons of "potential" ways to mitigate this without upgrading - some are covered above. But some of the things in that graphic (Like block with WAF) are really hard because of the number of OTHER msg lookup directives I can use. On you block ${jndi:ldap://} Do you ALSO block ${jndi:rmi://} ? I'm fairly certain I can exploit via RMI (I should test this and confirm - and research indicates it's config dependent). What about ${${lower:j}ndi:ldap//} etc. WAF bypass is fairly trivial.

Same with remote codebases - yeah you block the RCE, but I can suck arbitrary environment variables out via DNS. Guess what uses ENV variables a lot to pass in data - some potentially valuable - Spring boot in a container. I can look up your underlying os and java version.

Basically other than patching/disable log4j or disabling message lookups/jndi you have SOME vulnerability, and in a lot of code bases just turning off msg lookups makes logging break in some cases because there are legit use cases.

 

[Ian Andriechack]

I saw that Jon posted the class files.

Here is a list of vulnerable products by manufacturer - in case they are of interest:

log4shell/software at main · NCSC-NL/log4shell

Operational information regarding the log4shell vulnerabilities in the Log4j logging library. - NCSC-NL/log4shell

github.com

Also, a list of indicators (IP addresses) and so forth are available from the NCC group here:

Log4Shell: Reconnaissance and post exploitation network detection

Note: This blogpost will be live-updated with new information. NCC Group’s RIFT is intending to publish PCAPs of different exploitation methods in the near future – last updated Decembe…

research.nccgroup.com

They are in TruStar as well.

 

[Ron Culler]

Firewall/Security Appliance vendors guidance on impacted and cleared products. Most have also released signatures that should enable detection and blocking of this attack but only if you are able to do SSL Inspection on the appliance.

Fortinet has identified the following list of products that are impacted and those under active review:
Advisory ID FG-IR-21-245

PSIRT | FortiGuard Labs

www.fortiguard.com

SonicWALL has identified the following product as impacted and those under active review:
Advisory ID SNWLID-2021-0032
https://psirt.global.sonicwall.com/vuln-list

Cisco has identified an extensive list of products impacted and those under active review:
Advisory ID cisco-sa-apache-log4j-qRuKNEbd

Vulnerabilities in Apache Log4j Library Affecting Cisco Products: December 2021

Critical Vulnerabilities in Apache Log4j Java Logging Library On December 9, 2021, the following critical vulnerability in the Apache Log4j Java logging library affecting all Log4j2 versions earlier than 2.15.0 was disclosed: CVE-2021-44228: Apache Log4j2 JNDI features do not protect against...

www.cisco.com

Sophos has identified a list of products impacted and those under active review:

Cybersecurity as a Service Delivered | Sophos

We Deliver Superior Cybersecurity Outcomes for Real-World Organizations Worldwide with a Broad Portfolio of Advanced Security Products and Services.

www.sophos.com

Watchguard has identified a list of products impacted and those cleared:

Critical RCE Vulnerability in Log4J2 - Secplicity - Security Simplified

[Updated 13-12-2021: Additional information for WatchGuard customers] On Thursday, security researchers disclosed a critical, unauthenticated remote code execution (RCE) vulnerability in log4j2, a popular and widely used logging library for java applications. CVE-2021-44228 is a full 10.0 on the...

www.secplicity.org

PaloAlto has a list of cleared products

CVE-2021-44228 Impact of Log4j Vulnerabilities CVE-2021-44228, CVE-2021-45046, CVE-2021-45105, and CVE-2021-44832

Apache Log4j Java library is vulnerable to a remote code execution vulnerability CVE-2021-44228, known as Log4Shell, and related vulnerabilities CVE-2021-45046, CVE-2021-45105, and CVE-2021-44832. Log...

security.paloaltonetworks.com

 

[Nick Hansen]

N-able has posted this update on the status of our products

Apache Log4j Vulnerability – Updated 12 p.m. EST, December 15, 2021

Updated 2:30 p.m. EST, December 20, 2021 The Apache Software Foundation (ASF) has rolled out another update – version 2.17.0 – for its Java-based open-source logging library Log4j to ad…

status.n-able.com

--
Nick Hansen
Director, Product Management | N-able

 

[Ian Thornton-Trump CD]

Except that pretty much everyone is not running the LDAP server on 636 - it's almost always on a non-privileged port.

There are tons of "potential" ways to mitigate this without upgrading - some are covered above. But some of the things in that graphic (Like block with WAF) are really hard because of the number of OTHER msg lookup directives I can use. On you block ${jndi:ldap://} Do you ALSO block ${jndi:rmi://} ? I'm fairly certain I can exploit via RMI (I should test this and confirm - and research indicates it's config dependent). What about ${${lower:j}ndi:ldap//} etc. WAF bypass is fairly trivial.

Same with remote codebases - yeah you block the RCE, but I can suck arbitrary environment variables out via DNS. Guess what uses ENV variables a lot to pass in data - some potentially valuable - Spring boot in a container. I can look up your underlying os and java version.

Basically other than patching/disable log4j or disabling message lookups/jndi you have SOME vulnerability, and in a lot of code bases just turning off msg lookups makes logging break in some cases because there are legit use cases.

Except that pretty much everyone is not running the LDAP server on 636 - it's almost always on a non-privileged port.

There are tons of "potential" ways to mitigate this without upgrading - some are covered above. But some of the things in that graphic (Like block with WAF) are really hard because of the number of OTHER msg lookup directives I can use. On you block ${jndi:ldap://} Do you ALSO block ${jndi:rmi://} ? I'm fairly certain I can exploit via RMI (I should test this and confirm - and research indicates it's config dependent). What about ${${lower:j}ndi:ldap//} etc. WAF bypass is fairly trivial.

Same with remote codebases - yeah you block the RCE, but I can suck arbitrary environment variables out via DNS. Guess what uses ENV variables a lot to pass in data - some potentially valuable - Spring boot in a container. I can look up your underlying os and java version.

Basically other than patching/disable log4j or disabling message lookups/jndi you have SOME vulnerability, and in a lot of code bases just turning off msg lookups makes logging break in some cases because there are legit use cases.

Good solid analysis and I have a couple of of points to add. 1) There is clearly an architecture based mitigation with use of reverse proxying of the target host response to the malicious input - the attack then can be defanged almost completely. Port 389 LDAP over UDP and TCP would also (and would be a more effective) mitigation as would a more advanced firewall which would filter Protocols (rather than just ports) Blasting out LADAP into the Internet is kind of how we got here with the need for a "malicious LDAP" server connection from the target host. Although WAF as you said can be tricked - 99.9% of this is bot scanning with fairly easy finger printing - if your threat model is APT you are 100% right an actor can blast through pretty effectively. I'm not fussed as if your behind CloudFlare and other CDN's they are blocking it, EDR on your end points are all updated to kill it and kill any payload delivery and the big kicker here is not every Apache install is configured in a way to get "one shot exploited". Also keep in mind this is bot scanning and not a Worm so again I just feel we should not be at Defcon 5 like Wannacry, NotPetya or BadRabbit.