Parallel Database for OLTP and OLAP Just a survey article on materials on parallel database products and technologies for OLTP/OLAP applications. It mainly covers major commercial/academic efforts on developing parallel dbms to solve the...

I really want this blog to be a place to express my own ideas and thoughts, but I don't refuse reference other people's great ideas, especially when they are really helpful for me or potential readers. The following content is copied from a MSDN...

Jeff Dean gave a keynote Building Large Scale Information Retrieval Systems at WSDM 2009 . It's actually a presentation on how Google search engine evolves during the past 10 years. Here are my notes for this lecture. Part I - Overview of Search Engine...

Part I - .Net Networking Briefing All .Net networking facilities are build on top of Winsock2 subsystem, so concepts such as non-blocking I/O , async I/O and I/O multiplexing can all be applied to the managed world. .Net network framework just wrapped...

Behind Avatar's Fancy Special Visual Effect Since 12/18/2009, Avatar has attracted all people's eyes on this planet. I watched a 3D IMAX version of this film some time ago. After seeing those fancy special visual effect, I can't help wondering...

Database Technologies for Decision Support System Database technologies can be applied into two types of scenarios: - Transaction Processing ( OLTP ) - Analytic Processing , using statistical method( OLAP ) or machine/computational learning...

Parallel Programming - Using PVM PVM is an inactive direction in HPC community , but there are many lessons can be learned from its programming model, its architecture design/implementation and how/why it failed to be the dominate system....

Parallel Programming - Using MPI MPI is a message passing programming model standards[2] , it defines various Terms/Concepts , Data Structures and Function Signatures that are used to passing messages among computer processes. 1. Terms and...

Parallel Programming - Using OpenMP OpenMP is a parallel programming model for shared memory parallel computers. It's based on Fork-Join parallel execution pattern and is suitable for Data Parallel and Task Parallel applications. Fork-Join Pattern...

Parallel Programming - Using POSIX Threads Pthreads (a.k.a POSIX Threads), is another parallel programming model over Shared Memory Computers , which is categorized to Threads Based Model (the other is message passing based model). Pthreads...

Parallel Computing - An Introduction Parallel Computing is a form of computation in which many calculations are carried out simultaneously , operating on the principle that large problems can often be divided into smaller ones, which are...

Map/Reduce - in Functional Programming & Parallel Processing Perspectives Map/Reduce is a very popular term pair in today's technical community, mainly due to the popularity of its "inventor" - Google. But in fact, the terms and concepts...

Parallel DBMS V.S. Distributed DBMS 　Large Scale Data Intensive Computing is a hot topic today, many people starts to talk so called Parallel Database System and Distributed Database System technologies. But these two concepts seem very confusing...

Consistent Hashing - Theory & Implementation What's it? The consistent hashing comes from the solving of hot spot problem in Internet system, I.E., it comes from the distributed cache system. [1][2] The idea is simple and straight forward...

I/O Concept - Blocking/Non-Blocking VS Sync/Async 　　These concepts are discussed in the context of WinSock, but the basic ideas can be applied to other I/O types and also on other OS, such as Linux/Unix world as well. I - Blocking V.S. Non-Blocking...

Winsock I/O Model - Part II : Implementation In the previous post[6] , I summarized several scalable network I/O models in theory. In this article, I will give concrete code to show how to use each model to build a scalable network server...

Winsock I/O Model - Part I : Concept The basic steps to do windows socket programming are simple and straightforward: Server Side Initialize Winsock. Create a socket. Bind the socket. Listen on the socket for a client. Accept a connection from...

Inside Scalable I/O Model - In Sync & Async Way 　　The so called " I/O Model " describes how you write code to accomplish I/O tasks, such as connecting to a server, writing data to disk, receiving data from network etc. Programmatically...

Various concrete consistency models have been described in [1], now it's time to discuss how to implement these models. Consistency semantic is divided into two categories in [1] - "Coherence and Replication are very similar concepts and deal with the...

Part I - What's Data Consistency Model and Why Should We Care? Data Consistency Model - it is a Semantic Contract between a data storage system and its user. Here, data storage system may refer to hardware system ( for example : memory sub-system inDSM...

1. The Need for Logical Clock One of the challenges in distributed system is the lack of global time clocks, it's very hard to timestamp events is different processes and order them globally. To solve the " Time & Order of Events in Distributed...

Memory management is a core task in native world, careless usage of dynamic memory may cause the following problems: - 1. Heap Fragment , this will introduce performance penalty since it breaks data locality - 2. Memory Leak, it's a prgm correctness problem...

The " Windows System Programming "(3E) has a great appendix about Windows, Unix, C Library API comparison. It's obvious that this appendix lacks of many APIs in Memory, DLL and Security related areas, but it is still very helpful for referencing when...

Part I - General Steps and Principles 1. Define a Clear Goal - what's the purpose? to know how, to own components, to modify and extend? - results driven, what's the final outcome? 2. Know it as Client User - read user manual - get an overall big picture...

Conventionally, HPC/Parallel problems can be roughly divided into the following two categories[1][2]: - Data Parallel , these applications divides the input data into a number of completely independent parts. The same computation is undertaken on...