The best helpful hdfs file system commands (3 of 4)

-
dus-

hadoop fs -dus PATH

dus reports the sum of the file sizes in aggregate rather than individually.

expunge-

 hadoop fs -expunge

Empties the trash. If the trash feature is enabled, when a file is deleted, it is first moved into the temporary Trash/folder. The file will be permanently deleted from the Trash/folder only after user-configurable delay.

get -

hadoop -fs -get [-ignorecrc] [-crc] SRC
LOCASDST

Copies files to the local file system.

Comments

Post a Comment

Thanks for your message. We will get back you.

Popular Posts

The Growth of Machine Learning till TensorFlow

-
Image
The Internet and the vast amount of data are inspiration CEOs of big corporations started to use Machine learning. It is to provide a better experience to users. Let us take Amazon, online retail that uses Machine learning. The algorithm's purpose is to generate revenue. Based on user search data, the ML application provides information or insights. The other example is the advertising platform where Google is a leader in this line. Where it shows ads based on the user movements while surfing the web. Machine Learning These are just a few, but there are many in reality. Evolution of TensorFlow 1894 Francis Galton uses statistical regression to study inherited traits 1943 McCulloch and Pitts devise the first artificial neuron 1957 Frank Rosenblatt invents the perceptron 1963 Vapnik and Chervonenkis invent the Support Vector Machine algorithm 1974 Paul Werbos uses backpropagation to train a neural network 1982 John Hopfield demonstrates the Hopfield network 1998 Yann LeCunn trains a
Read more

How to Fix Python Syntax Errors Quickly

-
Image
In Python, you can call parsing errors Syntax errors. How to fix the syntax errors quickly, I can show in this post. Also, check Python Syntax Errors Cheat Sheet. Python Syntax Error. If you run the Python code, the interpreter first parses the logic. If any errors found, it is the developer's responsibility to fix them. Courtesy: Python documentation. >>> while True print('Hello world')   File "<stdin>", line 1     while True print('Hello world')                    ^ SyntaxError: invalid syntax How to Fix the Syntax Error First, check for the line number. In the above case, it is '1'. Then look for the symbol '^'. The issue is at the PRINT function. The reason is before the PRINT function you need a colon ':'. That is the reason, you are getting error. This is a syntax error. Find below the corrected logic: >>> while True : print('Hello world')   File "<stdin>", line 1     while Tr
Read more

7 AWS Interview Questions asked in Infosys, TCS

-
Image
I have shared 7 amazing AWS interview questions for your quick reference. 1). What is AWS? Amazon Web Services (AWS) provides on-demand computing resources and services in the cloud, with pay-as-you-go pricing. For example, you can run a server on AWS that you can log on to, configure, secure, and run just as you would a server that's sitting in front of you 2). What you can do with AWS? Store public or private data. Host a static website. These websites use client-side technologies (such as HTML, CSS, and JavaScript) to display content that doesn't change frequently. A static website doesn't require server-side technologies (such as PHP and ASP.NET). Host a dynamic website or web app. These websites include classic three-tier applications, with web, application, and database tiers. Support students or online training programs. Process business and scientific data. Handle peak loads. AWS Questions 3). What is the Security group? A security group acts as a virtual fi
Read more

How to Use the ps Command in Linux

-
Image
The real use of the ps command is to know the process details. In fact, along with process details, you would know many other parameters about the process. How many processes are running in a current Session you will get from the below examples. ps command Linux ps Command in Linux PID (Process Id) - It is the process identification number. TTY(Terminal name) - associated with this process TIME(Time) - the format is hh: mm: ss (Hours, Minutes, and Seconds). Cumulated time of that particular process. CMD (Executable Command name) I will show you an example, which is very basic, how it appears when you issue the  ps command. ps command You May Also Like: PS Command Options in Linux What is Slave/Master Terminal in Linux The pts/0 is the slave terminal. The  pts  means pseudo terminal slave. Raise your knowledge by bringing in these quick definitions.  LINUX - DIFFERENCE BETWEEN TTY AND PTS. In reality, the terminals are two types. One is ptmx (Master) and pts (Slave). The master termi
Read more

The best 5 differences of AWS EMR and Hadoop

-
Image
With Amazon Elastic MapReduce (Amazon EMR) you can analyze and process vast amounts of data. It does this by distributing the computational work across a cluster of virtual servers running in the Amazon cloud. The cluster is managed using an open-source framework called Hadoop. Amazon EMR - Elastic MapReduce, The Unique Features Amazon EMR has made enhancements to Hadoop and other open-source applications to work seamlessly with AWS. For example, Hadoop clusters running on Amazon EMR use EC2 instances as virtual Linux servers for the master and slave nodes, Amazon S3 for bulk storage of input and output data, and CloudWatch to monitor cluster performance and raise alarms. You can also move data into and out of DynamoDB using Amazon EMR and Hive. All of this is orchestrated by Amazon EMR control software that launches and manages the Hadoop cluster. This process is called an Amazon EMR cluster. What does Hadoop do... Hadoop uses a distributed processing architecture
Read more

Hyperledger Fabric: 20 Real Interview Questions

-
Image
I am practicing Hyperledger. This is one of the top listed blockchains. This architecture follows R3 Corda specifications. Sharing the interview questions with you that I have prepared for my interview. Though Ethereum leads in the real-time applications. The latest Hyperledger version is now ready for production applications. It has now become stable for production applications. The Hyperledger now backed by IBM. But, it is still an open source. These interview questions help you to read quickly. The below set of interview questions help you like a tutorial on Hyperledger fabric. Hyperledger Fabric Interview Questions 1). What are Nodes? In Hyperledger the communication entities are called Nodes. 2). What are the three different types of Nodes? - Client Node - Peer Node - Order Node The Client node initiates transactions. The peer node commits the transaction. The order node guarantees the delivery. 3). What is Channel? A channel in Hyperledger is the subnet of
Read more

How to Decode TLV Quickly

-
Image
In TLV, the format is Tag, Length, and Value. The TLV protocol needs this type of data. Here you will know how to decode TLV data.  According to IBM , TLV data is three parts. The tag tells what type of data it is. The length field denotes the length of the value. The Value-field denotes the actual value. Structure of TLV. TLV comprises three field values.  Tag Length Value EMV formulated different tags. They have their meanings. Usually, the Tag and Length together takes 1 to 4 bytes. The Best example for TLV. In the below example, you can find the sample TAG, LENGTH, and VALUE fields. [Tag][Value Length][Value] (ex. " 9F4005F000F0A001 ") where Tag Name =  9F40 Value Length (in bytes) =  05  Value (Hex representation of bytes. Example, "F0" – 1-byte) =  F000F0A001 In the above message, tag 9F40 has some meaning designed by EMV company. Here  you can find a list of EMV Tags. How to read the TLV Tag: 1 or 2 bytes Length: Length of the Value. F0-00-F0-A0-01 ==> 5 B
Read more

AWS Vs Azure Load Balancers Top Insights

-
Image
In AWS, you have three types of load-balancers. In the case of Azure, you have only two load balancers. This post tells you the comparison between these two. AWS Load Balancers: Application Load balancer Network Load balancer Classic Load balancers. Application Load Balancer Balancing of HTTP and HTTPS traffic and provides advanced request routing targeted at the delivery of modern application architectures, including microservices and containers.  Operating at the individual request level (Layer 7), Application Load Balancer routes traffic to targets within Amazon Virtual Private Cloud (Amazon VPC) based on the content of the request. Network Load Balancer Network Load Balancer is best suited for load balancing of Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and Transport Layer Security (TLS) traffic where extreme performance is required. Classic Load Balancer The Classic Load Balancer provides the basic load-balancing across multiple Amazon EC2 instances.  It op
Read more

QlikView Server vs Publisher Top Differences Really Useful to Your Project

-
Image
The differences between QlikView Server and publisher, I have given for your reference. These are related to internally how differ each other. QLIKVIEW SERVER The QVS is a server-side product.  It contains an  in-memory analytics engine , which handles all client/server communication between a QlikView client (Desktop, IE plugin, AJAX, or Mobile) and the server. The Administrator can have access to the management console to control all aspects of the server deployments (including security, clustering, distribution, etc.)  It also includes a web server to provide front-end access to the documents within. The web server’s user portal is known as Access Point. (It’s important to note that while the QVS contains its web server, one can utilize Microsoft IIS (Internet Information Server) for this purpose, too). The QVS handles client authorization against existing directory providers (Microsoft Active Directory, eDirectory) It also performs read and write to ACLs (access control lists) for
Read more

Top 10 SCALA Quiz Questions for Programmers

-
Image
Scala is an acronym for “Scalable Language” . This means that Scala grows with you. You can play with it by typing one-line expressions and observing the results. But you can also rely on it for large mission-critical systems , as many companies, including Twitter, LinkedIn, or Intel does. To some, Scala feels like a scripting language. Its syntax is a concise and low ceremony; its types get out of the way because the compiler can infer them. There are a REPL and IDE worksheets for quick feedback. Developers like it so much that Scala won the ScriptBowl contest at the 2012 JavaOne conference. At the same time, Scala is the preferred workhorse language for many mission-critical server systems. The generated code is on a par with Java’s and its precise typing means that many problems are caught at compile-time rather than after deployment. ✅ SCALA Quiz Link At the root, the language’s scalability is the result of a careful integration of object-oriented and functional language concepts
Read more