Microsoft has released a new report highlighting a new series of attacks targeting a toolkit called Kubeflow which is used for running machine learning operations on top of Kubernetes clusters.
The attacks first began in April of this year and have continued with the aim of installing a cryptocurrency miner on Kubernetes clusters that are exposed to the internet and run Kubeflow.
In a blog post, security research software engineer at the Azure Security Center, Yossi Weizman provided more details on Kubeflow and why nodes used for machine learning tasks are such an attractive target for cybercriminals, saying:
“Kubeflow is an open-source project, started as a project for running TensorFlow jobs on Kubernetes. Kubeflow has grown and become a popular framework for running machine learning tasks in Kubernetes. Nodes that are used for ML tasks are often relatively powerful, and in some cases include GPUs. This fact makes Kubernetes clusters that are used for ML tasks a perfect target for crypto mining campaigns, which was the aim of this attack.”
Misconfigured Kubeflow instances
Microsoft has tracked these attacks since they first started showing up online back in April. However, after the first attack wave, the cryptomining syndicate behind them switched from targeting general-purpose Kubernetes clusters to focus specifically on those using Kubeflow to run machine learning operations.
Based on findings from its initial investigation, the software giant now believes that misconfigured Kubeflow instances are the most likely point of entry for the attackers. This is likely the result of Kubeflow admins changing the toolkit’s default settings which exposed its admin panel online. By default, the Kubeflow management panel is only accessible from inside the Kubernetes cluster and not over the internet.
According to Weizman, a cryptomining syndicate is now actively scanning for these dashboards online. When found, the group deploys a new server image to Kubeflow clusters that runs a Monero cryptocurrency mining application called XMRig.
Server admins can check to see if their Kubeflow instances have been hacked by entering this command: kubectl get pods –all-namespaces -o jsonpath=”{.items[*].spec.containers[*].image}” | grep -i ddsfdfsaadfs. To prevent falling victim to these attacks, server admins should make sure that Kubeflow’s daashboard is not exposed to the internet.
Via ZDNet
