Apache
/
hadoop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

HADOOP-16764. Rewrite Python example codes using Python3 (#1762) 

(cherry picked from commit fd7de2b82a)
This commit is contained in:
Kengo Seki 2019-12-16 11:04:20 +09:00 committed by Akira Ajisaka
parent 2c25f80a9c
commit 5abf19a773
No known key found for this signature in database
GPG Key ID: C1EDBB9CA400FD50
2 changed files with 12 additions and 11 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
hadoop-common-project/hadoop-common/src/site/markdown/RackAwareness.md
View File

@ -64,7 +64,7 @@ rack and is unable to do so as there is only a single rack named
python Example python Example
-------------- --------------
```python ```python
#!/usr/bin/python #!/usr/bin/python3
# this script makes assumptions about the physical environment. # this script makes assumptions about the physical environment.
# 1) each rack is its own layer 3 network with a /24 subnet, which # 1) each rack is its own layer 3 network with a /24 subnet, which
# could be typical where each rack has its own # could be typical where each rack has its own
@ -94,9 +94,9 @@ for ip in sys.argv: # loop over lis
address = '{0}/{1}'.format(ip, netmask) # format address string so it looks like 'ip/netmask' to make netaddr work address = '{0}/{1}'.format(ip, netmask) # format address string so it looks like 'ip/netmask' to make netaddr work
try: try:
network_address = netaddr.IPNetwork(address).network # calculate and print network address network_address = netaddr.IPNetwork(address).network # calculate and print network address
print "/{0}".format(network_address) print("/{0}".format(network_address))
except: except:
print "/rack-unknown" # print catch-all value if unable to calculate network address print("/rack-unknown") # print catch-all value if unable to calculate network address
``` ```
bash Example bash Example

17
hadoop-tools/hadoop-streaming/src/site/markdown/HadoopStreaming.md.vm
View File

@ -416,27 +416,28 @@ To use Aggregate, simply specify "-reducer aggregate":
-output myOutputDir \ -output myOutputDir \
-mapper myAggregatorForKeyCount.py \ -mapper myAggregatorForKeyCount.py \
-reducer aggregate \ -reducer aggregate \
-file myAggregatorForKeyCount.py \ -file myAggregatorForKeyCount.py
The python program myAggregatorForKeyCount.py looks like: The python program myAggregatorForKeyCount.py looks like:
#!/usr/bin/python #!/usr/bin/python3
import sys; import sys
def generateLongCountToken(id): def generateLongCountToken(id):
return "LongValueSum:" + id + "\t" + "1" return "LongValueSum:" + id + "\t" + "1"
def main(argv): def main(argv):
line = sys.stdin.readline(); line = sys.stdin.readline()
try: try:
while line: while line:
line = line[:-1]; line = line[:-1]
fields = line.split("\t"); fields = line.split("\t")
print generateLongCountToken(fields[0]); print(generateLongCountToken(fields[0]))
line = sys.stdin.readline(); line = sys.stdin.readline()
except "end of file": except "end of file":
return None return None
if __name__ == "__main__": if __name__ == "__main__":
main(sys.argv) main(sys.argv)