I found torch.cuda.Stream() is manually defined in some open source code.
self.input_stream = torch.cuda.Stream()
self.model_stream = torch.cuda.Stream()
self.output_stream = torch.cuda.Stream()
On torch page, it says
You normally do not need to create one explicitly: by default, each device uses its own “default” stream.
Trying to understand why they had to define this manually. From the quick google search, there are lots of to use cuda.Stream() but no to use it.
0 Answer
I read in a StackOverflow post regarding it. Though it is also how to use.
But after reading it, all I got is:
Why/When use stream: To parallelize some computations.Best Practices to use the stream: To boost up some really extensive computational tasks through parallelizing. (Mostly; you are welcome to comment if any better answer is found.)Reference:
Streams are sequences of cuda kernels. Operations in different streams may run in parallel. I don't believe they have to use them. They are are just making the code more parallel and thus hopefully faster.
这家伙很懒，什么都没留下...