Recent questions tagged Hadoop

0 votes

451 views

1 answer

hadoop - What is the difference between -hivevar and -hiveconf?

From hive -h : --hiveconf <property=value> Use value for given property --hivevar <key=value> Variable subsitution to ... e.g. --hivevar A=B See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

794 views

1 answer

hadoop - Skip first line of csv while loading in hive table

Hello Friends, I created table in hive with help of following command - CREATE TABLE db.test ( fname STRING, ... first line. Thanks in advance. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

432 views

1 answer

hadoop - Connect from Java to Hive using JDBC

I'm trying to connect from Java to Hive server 1. I found a question time ago in this forum but it ... anybody know what's happening here? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

871 views

1 answer

hadoop yarn - How to exit spark-submit after the submission

When submitting spark streaming program using spark-submit(YARN mode) it keep polling the status and never exit ... JVM cannot create GC thread See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

489 views

1 answer

hadoop - How to execute Spark programs with Dynamic Resource Allocation?

I am using spark-summit command for executing Spark jobs with parameters such as: spark-submit --master yarn- ... in executing Spark programs. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

513 views

1 answer

hadoop - What is a container in YARN?

What is a container in YARN? Is it same as the child JVM in which the tasks on the nodemanager run or is it different? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

762 views

1 answer

hadoop - Does Hive have a String split function?

I am looking for a in-built String split function in Hive? e.g. if String is: A|B|C|D|E Then I ... a indexOf() and split() string functions. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

443 views

1 answer

hadoop - Difference between `yarn.scheduler.maximum-allocation-mb` and `yarn.nodemanager.resource.memory-mb`?

What is difference between yarn.scheduler.maximum-allocation-mb and yarn.nodemanager.resource.memory-mb? I see both ... these? Thank you. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

536 views

1 answer

hadoop - Datanode not starts correctly

I am trying to install Hadoop 2.2.0 in pseudo-distributed mode. While I am trying to start the ... hdfs.server.datanode.DataNode: SHUTDOWN_MSG: See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

524 views

1 answer

hadoop - Hive insert query like SQL

I am new to hive, and want to know if there is anyway to insert data into Hive table like we do in SQL. I ... way to append the data as in SQL? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

574 views

1 answer

hadoop - How does impala provide faster query response compared to hive

I have recently started looking into querying large sets of CSV data lying on HDFS using Hive and Impala. As I ... for the same data on HDFS? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

586 views

1 answer

hadoop - Error while configuring Apache Zeppelin on Windows 10

I get the following error while trying to install and configure Apache Zeppelin on Windows 10: org.apache.zeppelin. ... and ran example code. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

442 views

1 answer

hadoop - How to resolve Guava dependency issue while submitting Uber Jar to Google Dataproc

I am using maven shade plugin to build Uber jar for submitting it as a job to google dataproc cluster. ... watching this tag please help. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

488 views

1 answer

hadoop - How to Group mongodb - mapReduce output?

i have a query regarding the mapReduce framework in mongodb, so i have a result of key value pair from mapReduce ... {user15to20: 0 } .... See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

733 views

1 answer

hadoop - Why is Fetch task in Hive works faster than Map-only task?

It is possible to enable Fetch task in Hive for simple query instead of Map or MapReduce using hive hive.fetch.task. ... table data, isn't it? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

559 views

1 answer

hadoop - How can I execute Sqoop in Java?

I created a new Java project, then I added Library Sqoop and Hadoop. (The Libraries are "hadoop-core-1.1.1.jar, ... but it doesn't work for me. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

524 views

1 answer

hadoop - Spark Shell - __spark_libs__.zip does not exist

I'm new to Spark and I'm busy setting up a Spark Cluster with HA enabled. When starting a spark shell for testing ... /name> <value>slave1:23188</value> </property> <property> &lt...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

541 views

1 answer

hadoop - MapReduce shuffle/sort method

Somewhat of an odd question, but does anyone know what kind of sort MapReduce uses in the sort portion of shuffle/sort? ... ), but I'm not sure. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

451 views

1 answer

hadoop - Launch a mapreduce job from eclipse

I've written a mapreduce program in Java, which I can submit to a remote cluster running in distributed mode ... scripts which would be awesome! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

508 views

1 answer

hadoop - What is Lineage In Spark?

How lineage helps to recompute data? For example, I'm having several nodes computing data for 30 minutes each ... giving 15 minutes again? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

578 views

1 answer

hadoop - Unable to correctly load twitter avro data into hive table

Need your help! I am trying a trivial exercise of getting the data from twitter and then loading it ... (HiveIOExceptionHand lerChain.java:121) See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

755 views

1 answer

hadoop - get the current date and set it to variable in order to use it as table name in HIVE

I want to get the current date as YYMMDD and then set it to variable in order to use it as table name. ... the assignments are wrong. Any idea? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

515 views

1 answer

hadoop - Hive and regular expression

I am trying to filter all the ip adresses in a username. But this doesnt really work properly in my query: select ... Any idea how to fix it? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

452 views

1 answer

hadoop - HDFS Home Directory

I have setup a single node multi-user hadoop cluster. In my cluster, there is an admin user that is ... ownership to a particular user. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

450 views

1 answer

hadoop - MultipleTextOutputFormat alternative in new API

As it stands out MultipleTextOutputFormat have not been migrated to the new API. So if we need to choose an ... have with new mapreduce API ? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

658 views

1 answer

hadoop - Which files are ignored as input by mapper?

I'm chaining multiple MapReduce jobs and want to pass along/store some meta information (e.g. configuration or ... just a fixed limited list? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

479 views

1 answer

hadoop - MapReduce job Output sort order

i can see in my mapreduce jobs that the output of the reducer part is sorted by key .. so if i have ... throw some light why such a behavior See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

2.5k views

1 answer

hadoop - localhost: ERROR: Cannot set priority of datanode process 32156

I am trying to install hadoop on ubuntu 16.04 but while starting the hadoop it will give me following error ... error ? Thanks in advance. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags