This article guides you to install and test Apache Hadoop on your Debian system. The following steps will provide detailed description of the configuration process and verification methods.
Step 1: Install Java
Make sure that the system has Java 8 or higher installed. Install OpenJDK 8 using the following command:
sudo apt update sudo apt install openjdk-8-jdk
Verify installation:
java -version
Step 2: Download and decompress Hadoop
Download the latest version of Hadoop from the official website of Apache Hadoop and unzip it to the specified directory (for example /usr/local/hadoop
):
wget https://downloads.apache.org/hadoop/common/hadoop-3.3.1/hadoop-3.3.1.tar.gz tar -xzvf hadoop-3.3.1.tar.gz -C /usr/local/hadoop ``` (Please replace `hadoop-3.3.1` with the actual version number) **Step 3: Configure environment variables** Edit the `~/.bashrc` file and add the following environment variables: ```bash export HADOOP_HOME=/usr/local/hadoop export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
Make the changes take effect:
source ~/.bashrc
Step 4: Configure Hadoop configuration file
Modify the configuration file in the Hadoop directory:
- core-site.xml :
<configuration> <property> <name>fs.defaultFS</name> <value>hdfs://namenode:9000</value> </property> </configuration>
- hdfs-site.xml :
<configuration> <property> <name>dfs.replication</name> <value>3</value> </property> <property> <name>dfs.namenode.name.dir</name> <value>/usr/local/hadoop/dfs/name</value> </property> <property> <name>dfs.datanode.data.dir</name> <value>/usr/local/hadoop/dfs/data</value> </property> </configuration>
- mapred-site.xml :
<configuration> <property> <name>mapreduce.framework.name</name> <value>yarn</value> </property> </configuration>
- yarn-site.xml :
<configuration> <property> <name>yarn.nodemanager.aux-services</name> <value>mapreduce_shuffle</value> </property> <property> <name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name> <value>org.apache.hadoop.mapred.ShuffleHandler</value> </property> </configuration>
Step 5: Format HDFS
On the NameNode node, execute the following command to format HDFS:
hdfs namenode -format
Step 6: Start Hadoop Service
Start NameNode and DataNode on the NameNode node:
start-dfs.sh
Start YARN on the ResourceManager node:
start-yarn.sh
Step 7: Local Mode Testing
Switch to Hadoop user:
su - hadoop
Create input directories and files:
mkdir ~/input vi ~/input/data.txt
Enter test data (for example "Hello World", "Hello Hadoop"), save and exit.
Run WordCount example:
hadoop jar /usr/local/hadoop/share/hadoop/mapreduce/hadoop-mapreduce-examples-*.jar wordcount ~/input/data.txt ~/output ``` (Please adjust it according to the actual jar package file name) View the results: ```bash ls ~/output cat ~/output/part-r-00000
The correct output indicates that Hadoop local mode is running successfully. Please note that the above steps assume that you are testing in a stand-alone environment. For cluster environments, corresponding configuration modifications are required. Be sure to refer to the official Hadoop documentation for more detailed and latest configuration information.
The above is the detailed content of How to test Debian Hadoop. For more information, please follow other related articles on the PHP Chinese website!

Hot AI Tools

Undress AI Tool
Undress images for free

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics

PHP code can be executed in many ways: 1. Use the command line to directly enter the "php file name" to execute the script; 2. Put the file into the document root directory and access it through the browser through the web server; 3. Run it in the IDE and use the built-in debugging tool; 4. Use the online PHP sandbox or code execution platform for testing.

Updating the Tomcat version in the Debian system generally includes the following process: Before performing the update operation, be sure to do a complete backup of the existing Tomcat environment. This covers the /opt/tomcat folder and its related configuration documents, such as server.xml, context.xml, and web.xml. The backup task can be completed through the following command: sudocp-r/opt/tomcat/opt/tomcat_backup Get the new version Tomcat Go to ApacheTomcat's official website to download the latest version. According to your Debian system

To develop a complete Python Web application, follow these steps: 1. Choose the appropriate framework, such as Django or Flask. 2. Integrate databases and use ORMs such as SQLAlchemy. 3. Design the front-end and use Vue or React. 4. Perform the test, use pytest or unittest. 5. Deploy applications, use Docker and platforms such as Heroku or AWS. Through these steps, powerful and efficient web applications can be built.

The garbled problem in Java Chinese is mainly caused by inconsistent character encoding. The repair method includes ensuring the consistency of the system encoding and correctly handling encoding conversion. 1.Use UTF-8 encoding uniformly from files to databases and programs. 2. Clearly specify the encoding when reading the file, such as using BufferedReader and InputStreamReader. 3. Set the database character set, such as MySQL using the ALTERDATABASE statement. 4. Set Content-Type to text/html;charset=UTF-8 in HTTP requests and responses. 5. Pay attention to encoding consistency, conversion and debugging skills to ensure the correct processing of data.

For good reason, Blockdag focuses on buyer interests. Blockdag has raised an astonishing $265 million in 28 batches of its pre-sales As 2025 approaches, investors are steadily accumulating high-potential crypto projects. Whether it’s low-cost pre-sale coins that offer a lot of upside, or a blue chip network that prepares for critical upgrades, this moment provides a unique entry point. From fast scalability to flexible modular blockchain architecture, these four outstanding names have attracted attention throughout the market. Analysts and early adopters are watching closely, calling them the best crypto coins to buy short-term gains and long-term value now. 1. BlockDag (BDAG): 7 days left

Linux system restricts user resources through the ulimit command to prevent excessive use of resources. 1.ulimit is a built-in shell command that can limit the number of file descriptors (-n), memory size (-v), thread count (-u), etc., which are divided into soft limit (current effective value) and hard limit (maximum upper limit). 2. Use the ulimit command directly for temporary modification, such as ulimit-n2048, but it is only valid for the current session. 3. For permanent effect, you need to modify /etc/security/limits.conf and PAM configuration files, and add sessionrequiredpam_limits.so. 4. The systemd service needs to set Lim in the unit file

Custom Laravel user authentication logic can be implemented through the following steps: 1. Add additional verification conditions when logging in, such as mailbox verification. 2. Create a custom Guard class and expand the authentication process. Custom authentication logic requires a deep understanding of Laravel's authentication system and pay attention to security, performance and maintenance.

ServiceWorker implements offline caching by intercepting network requests and providing pre-cache resources. The specific steps include: 1) Register ServiceWorker and check browser support; 2) Define cache policies and pre-cache resources in the sw.js file; 3) Pre-cache resources using install event and decide to obtain resources from the cache or network in the fetch event; 4) Pay attention to version control, cache policy selection and debugging skills; 5) Optimize cache size, process dynamic content, and ensure that scripts are loaded through HTTPS.
