Recent questions tagged mapreduce

0 votes

465 views

1 answer

mapreduce - Write to multiple tables in HBASE

I have a situation here where I need to write to two of the hbase tables say table1,table 2. Whenever ... should not include co-processors. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

390 views

1 answer

mapreduce - hadoop converting to and breaking ARC format

I am trying to parse data from commoncrawl.org using hadoop streaming. I set up a local hadoop to ... eventually run on AWS ElasticMapReduce. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

389 views

1 answer

mapreduce - How to handle id generation on a hadoop cluster?

I am building a dictionary on a hadoop cluster and need to generate a numeric id for each token. How should I do it? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

367 views

1 answer

mapreduce - How to change the structure of MongoDB's map-reduce results?

When I'm running Map-Reduce on a Mongo database, I usually get results similar to the following: { _id: < ... withing a map or reduce function? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

540 views

1 answer

mapreduce - Wordcount program is stuck in hadoop-2.3.0

I installed hadoop-2.3.0 and tried to run wordcount example But it starts the job and sits idle hadoop@ubuntu:~ ... hadoop-2.3.0 Please suggest. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

339 views

1 answer

mapreduce - Configure Map Side join for multiple mappers in Hadoop Map/Reduce

I have a question about configuring Map/Side inner join for multiple mappers in Hadoop. Suppose I have two ... pairs of files simultaneously? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

422 views

1 answer

mapreduce - "merge" view collation into useful output in CouchDB

When doing a "join" in CouchDB, you can use view collation to group the records together. For example, having two ... is how exactly to do this? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

492 views

1 answer

mapreduce - "merge" view collation into useful output in CouchDB

When doing a "join" in CouchDB, you can use view collation to group the records together. For example, having two ... is how exactly to do this? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

434 views

1 answer

mapreduce - Implementation of an ArrayWritable for a custom Hadoop type

How do I define an ArrayWritable for a custom Hadoop type ? I am trying to implement an inverted index in ... values in the Reduce method See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

479 views

1 answer

mapreduce - Hadoop namenode : Single point of failure

The Namenode in the Hadoop architecture is a single point of failure. How do people who have large Hadoop clusters ... the primary one fails ? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

440 views

1 answer

mapreduce - Fast way to find duplicates on indexed column in mongodb

I have a collection of md5 in mongodb. I'd like to find all duplicates. The md5 column is indexed. Do you ... { out.duplicates.insert(obj) }); See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

519 views

1 answer

mapreduce - What are SUCCESS and part-r-00000 files in hadoop

Although I use Hadoop frequently on my Ubuntu machine I have never thought about SUCCESS and part-r-00000 files ... just a randomly defined? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

412 views

1 answer

mapreduce - How does Hadoop decide how many nodes will perform the Map and Reduce tasks?

I'm new to hadoop and I'm trying to understand it. Im talking about hadoop 2. When I have an input file ... nodes will do the reduce tasks? See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

326 views

1 answer

mapreduce - Hadoop: No Such Method Exception

I have written a MapReduce program, code is below: import java.io.IOException; import java.util.Iterator; import org ... I rectify it? Thanks! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

342 views

1 answer

mapreduce - What is Keyword Context in Hadoop programming world?

What exactly is this keyword Context in Hadoop MapReduce world in new API terms? Its extensively used to write ... for your time and help. See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

341 views

1 answer

mapreduce - Hadoop: how to access (many) photo images to be processed by map/reduce?

I have 10M+ photos saved on the local file system. Now I want to go through each of them to analyze the ... terms of the above question, Thanks! See Question&Answers more detail:os...

asked Oct 24, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

450 views

1 answer

mapreduce - Output a list from a Hadoop Map Reduce job using custom writable

I'm trying to create a simple map reduce job by changing the wordcount example given by hadoop. I'm trying ... job.waitForCompletion(true); } } See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

421 views

1 answer

mapreduce - About Hadoop/HDFS file splitting

Want to just confirm on following. Please verify if this is correct: 1. As per my understanding when we copy ... available to a single mapper. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

864 views

1 answer

mapreduce - Hadoop WordCount example stuck at map 100% reduce 0%

[hadoop-1.0.2] → hadoop jar hadoop-examples-1.0.2.jar wordcount /user/abhinav/input /user/abhinav/output ... . What could be causing this? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

349 views

1 answer

mapreduce - Map-Reduce performance in MongoDb 2.2, 2.4, and 2.6

I've found this discussion: MongoDB: Terrible MapReduce Performance. Basically it says try to avoid Mongo's MR ... to effectively use indexes? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

489 views

1 answer

mapreduce - Map Reduce with F# agents

After playing with F# agents I tried to do a map reduce using them. The basic structure I use is: map supervisor ... ) mp.Status() mp.Stop() See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

495 views

1 answer

mapreduce - hadoop: difference between 0 reducer and identity reducer?

I am just trying to confirm my understanding of difference between 0 reducer and identity reducer. 0 reducer means ... will still take place? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

658 views

1 answer

mapreduce - Split size vs Block size in Hadoop

What is relationship between split size and block size in Hadoop? As I read in this, split size must be n ... between split size and block size? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

493 views

1 answer

mapreduce - Hadoop input split size vs block size

I am going through hadoop definitive guide, where it clearly explains about input splits. It goes like Input splits ... lesser than block size? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

676 views

1 answer

mapreduce - What is the use of grouping comparator in hadoop map reduce

I would like to know why grouping comparator is used in secondary sort of mapreduce. According to the definitive ... it. Thanks in advance See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

478 views

1 answer

mapreduce - How does Hadoop perform input splits?

This is a conceptual question involving Hadoop/HDFS. Lets say you have a file containing 1 billion lines. And ... val parameters? Thanks, Deepak See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

535 views

1 answer

mapreduce - Find all duplicate documents in a MongoDB collection by a key field

Suppose I have a collection with some set of documents. something like this. { "_id" : ObjectId(" ... "bar" appears 3 times. See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

0 votes

537 views

1 answer

mapreduce - Remove Duplicates from MongoDB

hi I have a ~5 million documents in mongodb (replication) each document 43 fields. how to remove duplicate document ... can solve the problem ? See Question&Answers more detail:os...

asked Oct 17, 2021 in Technique[技术] by 深蓝 (71.8m points)

Categories

Just Browsing Browsing

Most popular tags