ࡱ > Q T P bjbj @( = = = = = Q Q Q Q 4 , Q ?
L? N? N? N? N? N? N? $ A yD 4 r? = r? = = ? ~ ~ ~ = = L? ~ L? ~ ~ : = Q ; " 8? ? 0 ? ; D D D = = D = = < ~ r? r? ~ ? D : CS259: Advanced Parallel Processing
As an Introduction, here are some Interesting Facts:
- No matter how much computer power people have, they always want more
Computer capacity exhibits a highly "elastic" demand
Self-reinforcing cycle:
More powerful computers make new applications possible
New applications require even more powerful computers
=> The Demand for computational power will always exceed the supply <=
More Accuracy, Data, Kinds of Data, Speed, Interconnectivity
- The Current Best Known Approach to Achieving Cost-Effective and
Energy-Efficient High Performance Computing is Parallel Processing
=> The Software Field is being Reinvented (or dragged into parallelism by HW)
Hardware Manufacturers are forced to utilize multi-, many-core designs
Unlike ILP and SuperScalar parallelism which was transparent to SW,
Multi- and Many-Core HW directly exposes parallelism to SW
=> To be a good programmer, you will need to know about parallel processing
Note the Course Title: Parallel Processing; not Parallel Programming
Parallelism can occur in Hardware and Software
HW Computations and SW Computations are Interchangeable
The two are synergistically interrelated and dependent upon each other
Analogy to a High-Performance Race Car Driver
Car is Hardware; Driver is Software
The System is Car and Driver
To write optimal (parallel) SW, CS experts need to understand (parallel) HW
Therefore, this class is a combination of HW and SW
Focus of the Class is on Parallelism in General
We will study Concepts, Algorithms and Potential Problems of Parallelism
Focus is *not* on any one particular programming language or paradigm
The Parallel Processing Field is currently in a state of Rapid Evolution.
Parallel Processing Languages and Paradigms are in flux, but
Fundamental Concepts, Algorithms (and Problems) will Endure
General Programming Background should be sufficient for this class
We will write many short programs as opposed to a few long programs
Informal Class Survey
How Many Students in Class Have:
- Taken CS159 Parallel Processing
- Taken CS247 Computer Architecture (Pipelining & ILP)
- Programming Backgrounds in: Fortran C Java
- Prior Knowledge or Experience with:
Multi-Threading, Concurrency, Parallelism
Deadlocks
Race Conditions
Mutual Exclusion, Semaphores
Memory Interleaving
Cache Memory, Cache Coherency
Virtualization
- Access to a Multi-Core PC, Cluster, Grid, Cloud or Supercomputer
- A Particular Topic or Goal in Mind for this Course ?
$ e f g
2
C
D
a
b
u
v
ǵǧvkkk]]k]k]J $hX h%- 5B* CJ( OJ QJ ph P hX hq^ 5CJ( OJ QJ hX 5CJ( OJ QJ 'hX h%- 56B*CJ( OJ QJ ph !hX 56B*CJ( OJ QJ ph hX h%- 5CJ aJ hX h%- 5CJ( OJ QJ "hx 5B*CJ0 OJ QJ aJ0 ph (hX h 5B*CJ0 OJ QJ aJ0 ph "h@ 5B*CJ0 OJ QJ aJ0 ph "hty 5B*CJ0 OJ QJ aJ0 ph $ Y , e f 2
v
w
U
`d 1$ ]`gdb( d 1$ ]^gdX
d 1$ ]gd%- d 1$ ]gdX
& F d 1$ ]gdX $`d 1$ ]`a$gdxx v
w
& ( ) ʵʪ~kWDkW6 hX hE 5CJ( OJ QJ $h7k h 5B*CJ( OJ QJ ph 'h7k h 56B*CJ( OJ QJ ph $h7k h%- 5B*CJ( OJ QJ ph hX h%- 5CJ( OJ QJ hX h 5CJ( OJ QJ h%- hZ 5CJ OJ QJ aJ h%- 5CJ$ OJ QJ (h hb( 5B*CJ OJ QJ aJ ph hb( 5B*CJ( OJ QJ ph $h hb( 5B*CJ( OJ QJ ph $hb( hb( 5B*CJ OJ QJ ph U ' X
1
Q
O `d 1$ ]`gdJ
& F `d 1$ ]`gdX
`d 1$ ]`gd `d 1$ ]`gdX
`d 1$ ]`gd
& F `d 1$ ]`gdZ
d 1$ ]gd%-
`d 1$ ]`gdb( ) X t
0
1
F
I
Q
N P mbR h_ 5B*CJ( OJ QJ ph hJ 5CJ( OJ QJ $h_ h_ 5B*CJ( OJ QJ ph hX h
5CJ( OJ QJ hX h 5CJ OJ QJ aJ hX h8 56CJ( OJ QJ hX h8 5CJ( OJ QJ hX h 5CJ( OJ QJ hX hE 5CJ( OJ QJ hX 5CJ( OJ QJ hX h%5 5CJ( OJ QJ hX h 5CJ( OJ QJ O ! ` # $ G H u `hd 1$ ]`^hgdb( `d 1$ ]``gdx
`d 1$ ]`gdx
& F `d 1$ ]`gdX `hd 1$ ]`^hgdJ $`d 1$ ]`a$gdJ
`d 1$ ]`gdZ
`d 1$ ]`gd `d 1$ ]`gdJ
! $ 0 u ! zgTF8 hX hZ 5CJ( OJ QJ hr hJ 5OJ QJ aJ $hJ hJ 5B*CJ0 OJ QJ ph $hJ hf b 5B*CJ0 OJ QJ ph hX hZ 5CJ OJ QJ aJ hX h%- 5CJ( OJ QJ hJ 5CJ( OJ QJ h_ 5CJ( OJ QJ $h_ h 5B*CJ( OJ QJ ph hX h 5CJ( OJ QJ hX h
5CJ( OJ QJ h_ 5B*CJ( OJ QJ ph $h_ h
5B*CJ( OJ QJ ph ! " # $ G H P V W 0 2 < = ? ] _ ξγti^Pi hX hq^ 5CJ( OJ QJ h7k 5CJ( OJ QJ hG4 5CJ( OJ QJ h3 5CJ( OJ QJ hX h3 5CJ( OJ QJ hz 5CJ( OJ QJ hG4 hG4 5CJ OJ QJ aJ hb( 5CJ( OJ QJ hty 5CJ( OJ QJ hx hb( 5CJ OJ QJ aJ hx 5CJ( OJ QJ hx hx 5CJ OJ QJ aJ hf b 5CJ( OJ QJ hJ 5CJ( OJ QJ H = Q o ~ { `pP P 1$ ]`^p`gdG4 `@ P P 1$ ]`^@ gdz `P P 1$ ]`^`gdz `pP P 1$ ]`^p`gdz `P P 1$ ]`gdx `hd 1$ ]`^hgdG4 `d 1$ ]`^gdb( _ n o ~ hq^ 5CJ$ OJ QJ hX hs6 5CJ( OJ QJ hs6 5CJ( OJ QJ hX h3 5CJ( OJ QJ hX hZ 5CJ( OJ QJ hX hf b 5CJ( OJ QJ hG4 hs6 5CJ OJ QJ aJ hr 5CJ( OJ QJ hq^ 5CJ( OJ QJ hJ 5CJ( OJ QJ ~ `P P 1$ ]`gdb( `P P 1$ ]`gdz > 0 P 0:p = /! "#$% Dp f 6 6 6 6 6 6 6 6 6 v v v v v v v v v 6 6 6 6 6 6 > 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 h H 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 2 0 @ P ` p 2 ( 0 @ P ` p 0 @ P ` p 0 @ P ` p 0 @ P ` p 0 @ P ` p 0 @ P ` p 8 X V ~ OJ QJ _HmH nH sH tH @ ` @ N o r m a l CJ ^J _HmH sH tH D A D
D e f a u l t P a r a g r a p h F o n t R i R 0 T a b l e N o r m a l 4
l 4 a ( k (
0 N o L i s t 4 4 F o o t e r
4 4 H e a d e r
H & H F o o t n o t e R e f e r e n c e CJ EH aJ : ":
F o o t n o t e T e x t CJ . ) 1. P a g e N u m b e r R BR v i e w g r a p h t e x t d 5CJ$ OJ QJ ^J PK ! [Content_Types].xmlj0Eжr(Iw},-j4 wP-t#bΙ{UTU^hd}㨫)*1P' ^W0)T9<l#$yi} ;~@(Hu* Dנz/0ǰ $ X3aZ,D0j~3߶b~i>3\`?/[G\!-Rk.sԻ..a濭? PK ! ֧ 6 _rels/.relsj0 }Q%v/C/} (h"O
= C?hv=Ʌ%[xp{۵_Pѣ<1H0ORBdJE4b$q_ 6LR7`0̞O,En7Lib/Se е PK ! ky theme/theme/themeManager.xml M
@}w7c(Ebˮ CAǠҟ7՛K
Y,
e.|,H ,lxɴIsQ}#Ր ֵ+!,^$j=GW)E+&
8 PK ! P theme/theme/theme1.xmlYOo6w toc'vuر-Mn iP@I}úama[إ4:l ЯGRX^6؊>$ !)O^rC$y@ /yH*)UDb`}"qۋJחX^)I`nEp)liV[]1M<OP6r=zgbIguSe bORD۫qu gZo~ٺlAplxpT0+[}`j zA V2Fi@qv֬5\|ʜ̭NleXd sjcs7f
W+Ն7` g ȘJj|h(KD-
dXiJ؇(x$( :;˹! I_TS 1?E ??ZBΪmU/ ?~xY'y5g&/ɋ>GMGeD3Vq%'#q$8K )fw 9:ĵ
x}rxwr:\TZaG*y8IjbRc|XŻǿI
u3KGnD 1NIBs
RuK>V.E L+M2#'fi ~V vl{u8z H
* :( W ☕
~JTe\O*tHGHY }KNP*ݾ˦ TѼ9/#A7qZ $*c?qUnwN%Oi4 =3ڗP
1Pm \\9Mؓ2aD];Yt\[x]}Wr|]g-
eW
)6-rC Sj
id DЇAΜIqbJ#x꺃 6k#ASh&ʌt(Q% p%m&]caS l=X\P1Mh9MVdDAaVB[݈fJíP|8 քAV^f
Hn- "d>znNJ ة>b&2vKyϼD:,AGm\nziÙ .uχYC6OMf3or$5N HT[XF64T,ќM0E)`#5XY`פ ;%1U٥m;R>QD DcpU'&LE/pm%]8firS4d 7y\`JnίI R 3U~7+#m qBiDi*L69mY&iHE=(K&N!V.KeLDĕ{D vEꦚdeNƟe(MN9ߜR6&3(a/DUz<{ˊYȳV)9Z[4^n5!J?Q3eBoCM m<.vp IYfZY_p[=al-Y}Nc͙ŋ4vfavl'SA8|*u{-ߟ0%M07% < ҍ PK !
ѐ ' theme/theme/_rels/themeManager.xml.relsM
0wooӺ&݈Э5
6?$Q
,.aic21h:qm@RN;d`o7gK(M&$R(.1r'JЊT8V"AȻHu}|$b{ P8g/ ]QAsم(#L[ PK- ! [Content_Types].xmlPK- ! ֧ 6 + _rels/.relsPK- ! ky theme/theme/themeManager.xmlPK- ! P theme/theme/theme1.xmlPK- !
ѐ ' theme/theme/_rels/themeManager.xml.relsPK ]
( v
) ! _
U O H ~
8 @ 0 (
B
S ?
Y Y v " 0 2 = = Q Q ] _ n }
K ? A B C D E F G I J K L M N O R S V R o o t E n t r y F 0U D a t a
1 T a b l e D W o r d D o c u m e n t
@( S u m m a r y I n f o r m a t i o n ( @ D o c u m e n t S u m m a r y I n f o r m a t i o n 8 H M s o D a t a S t o r e Х I Q C D U 4 O D J V = = 2 Х I t e m
P r o p e r t i e s U C o m p O b j
y
F' Microsoft Office Word 97-2003 Document
MSWordDoc Word.Document.8 9q