Top Key Architecture Components in HIVE
5 architectural components present in Hadoop Hive: Shell: allows interactive queries like MySQL shell connected to a database – Also supports web and JDBC clients Driver: session handles, fetch, execute Compiler: parse, plan, optimize Execution engine: DAG of stages (M/R, HDFS, or metadata) Metastore: schema, location in HDFS, SerDe
Data Mode of Hive:
– Also, list: map (for JSON-like data)
HIVE Meta Store
– With custom SerDe, can use arbitrary format
Data Mode of Hive:
- Tables
– Also, list: map (for JSON-like data)
- Partitions
- Buckets
HIVE Meta Store
- Database: namespace containing a set of tables
- Holds table definitions (column types, physical layout)
- Partition data
- Uses JPOX ORM for implementation; can be stored in Derby, MySQL, many other relational databases
- Warehouse directory in HDFS
- Tables stored in subdirectories of warehouse
- Actual data stored in flat files
– With custom SerDe, can use arbitrary format
Comments
Post a Comment
Thanks for your message. We will get back you.