In this assignment, you will be designing and implementing MapReduce algorithms for a variety of common data processing tasks. Problem 4 The relationship "friend" is often symmetric, meaning that if I am your friend, you are my friend. Implement a MapReduce algorithm to check whether this property holds. Generate a list of all non-symmetric friend relationships.
Map Input
The input is a 2 element list: [personA, personB]
personA: Name of a person formatted as a string
personB: Name of one of personA’s friends formatted as a string
This implies that personB is a friend of personA, but it does not imply that personA is a friend of personB. Reduce Output
The output should be the (person, friend) and (friend, person) tuples for each asymmetric friendship.
Note however that only one of the (person, friend) or (friend, person) output tuples will exist in the input. This indicates friendship asymmetry.
You can test your solution to this problem using friends.json:
python asymmetric_friendships.py friends.json
You can verify your solution against asymmetric_friendships.json.
import MapReduce import sys """ Word Count Example in the Simple Python MapReduce Framework """ mr = MapReduce.MapReduce() # ============================= # Do not modify above this line def mapper(record): # key: document identifier # value: document contents person = record[0] #mr.emit_intermediate(person,1) pair = tuple(sorted(record)) mr.emit_intermediate(pair , 1) def reducer(pair, list_of_values): # key: word # value: list of occurrence counts #mr.emit((person,len(list_of_values)) ) if len(list_of_values) == 1: mr.emit((pair[0], pair[1]) ) mr.emit((pair[1], pair[0]) ) # Do not modify below this line # ============================= if __name__ == '__main__': inputdata = open(sys.argv[1]) mr.e xecute(inputdata, mapper, reducer)
View profile Serhii Dymenko
Leave a Comment