Is There Life Outside Transactions - Andreas Reuter.pdf

盖国强

128

5页

0次

2024-01-21

5墨值下载

Is There Life Outside Transactions?

Writing the Transaction Processing Book

Andreas Reuter

European Media Laboratory

Schlosswolfsbrunnenweg 33

D-69118 Heidelberg

+49-6221-533200

reuter@eml.org

ABSTRACT

In this article I will reflect on the writing of “Transaction

Processing – Concepts and Techniques” [1], which

appeared at Morgan Kaufmann Publishers in 1992. The

process of writing had many aspects of a typical software

project: In the end, the book was more than twice as thick

as we had planned, it covered only ¾ of the material that

we wanted to cover, and completing it took much longer

than we had anticipated. Nevertheless, it was a moderate

success and served as a basic reference for many

developers in the industry for at least 10 years after its

publication. It was translated to Chinese and Japanese, and

occasionally one still finds references to it – despite the fact

that (apart from simple bug fixes) there has been no

technical update of the material, and the book deals with

“outdated” subjects like transaction processing and

client/server architectures.

Categories and Subject Descriptors

K.2 [History of Computing]: People

General Terms

Performance, Design, Reliability, Standardization.

Keywords

Transaction processing, fault tolerance, software

architecture.

1. INTRODUCTION

In 1986 Jim had signed a contract with a seminar organizer

for teaching a one-week course on transaction processing in

spring 1987. The course was to take place in Berlin, and

because he did not want to teach the full five-day load all

by himself, he invited me to share some of it, assuming that

university professors have most – if not all – of the course

material ready for delivery on short notice. For the

following eight months we worked on preparing the slides.

The “plan” guiding the process was a list of chapter

headings that each of us would work on, with each chapter

representing a 90 min lecture. There was very little

communication along the way: A phone call now and then,

but no exchange of drafts or anything like that. Exchanging

emails between a university and a (closed) company

environment – Tandem in our case – was not something

you could simply do, and slides were either drawn by hand

or printed and then copied onto transparencies

. Two weeks

before the course started we sent a complete set of paper

copies of our 980 foils to the organizer, who turned them

into handouts for the participants. Delivering them all

would have been quite a challenge, but as soon as we found

out that the participants were very knowledgeable

developers with good ideas and tricky questions (I

remember Bill Laing and Paul Shrager being among them),

Jim decided to change the course structure completely.

Instead of following the table of contents in the handouts

we focused on what the participants were interested in and

skipped those parts that nobody wanted to hear about. For

example, we discussed a design where (for reliability) the

requests are sent to two completely independent systems

and executed on both. The question was, under which

conditions one could guarantee that the transactions are

serialized in the same order on both systems. So it was a

very lively, highly interactive course. I do not know how

much the participants took home, but the two presenters

definitely learned a lot.

2. ROBERT BURNS WAS RIGHT

When we reviewed the course, Jim observed that the

students’ questions had produced something that our initial

organization of the material had not suggested: A systems-

oriented perspective on transaction processing. So instead

of describing TP technology in isolation, and then

describing databases, networking, programming issues, etc.

we presented transactional concepts as some kind of

unifying framework for all layers of a system, from the

operating system all the way up to the applications and the

user interfaces. Jim went on saying that there were no

textbooks taking that integrative approach, and that it

should not be too hard to turn our 980 foils into text. We

estimated that on average two slides would transform into

one page of prose (including tables and figures), so that we

would have to write ca. 500 pages – 250 pages per person.

Doing this within a year seemed quite reasonable at the

In 1987 Microsoft bought the company that had produced the

precursor to Powerpoint, but it took a couple of years for that to

impact our teaching habits.

SIGMOD Record, June 2008 (Vol. 37, No. 2)

time, given that we had most of the material already. This

is how the whole thing started.

Before we actually tried to implement it, the initial plan

seemed to make perfect sense: The major portion of the

work had already been completed by putting all the

technical material on the slides – or so we thought. We

would only have to convert bullets into sentences, redo

some of the figures, add a chapter drawing all the details

into the grand picture of transaction-oriented systems,

compile a list of references – and be done! It was the kind

of plan that everybody will enthusiastically agree to at the

end of a meeting, so they can get on with their real work. In

our case it was the review meeting after the course, and

neither Jim nor I had a clear idea of how to implement it

after we got back from Berlin. However, with the best of

intentions we agreed on producing text from the slides

some time soon.

With no deadline at all and many other things to do, we

made very little progress in turning the foils into prose – in

fact, we did not make any progress at all. I used the

material for a variety of courses I taught at the university,

extending and changing it as new algorithms, new systems

etc. became available. Jim did the same, teaching

transaction processing at Stanford, but we still were just

using and updating the slides; no prose was being produced

as a result of the teaching activities. The only new type of

content that proved useful when – much later – we actually

wrote the book was a rapidly growing number of problems

and exercises related to the various topics that were

covered in the foils. Those problems were specifically

created for the university courses; they had not been part of

the Berlin seminar.

That was the situation in 1987, and it did not change in

1988, or in 1989. In the fall of 1989 we discussed the

project and found that the original plan had been a failure.

It was obvious that if we wanted to get anything written, we

would have to hide in some remote, quiet and pleasant

place, equipped with PCs, printer, toner, with easy access

to good food, and spend all our time typing – well, most of

it. We figured that three months should be enough to

produce a complete first draft of the book, the polishing of

which could be done later, when we were back in our

normal habitats. After some lengthy and careful

deliberation it was decided to rent a house in a small village

in Tuscany named Ripa (near Carrara) and spend February

through April of 1990 there.

This time we got it partially right: At the end of April we

had about 600 pages of text, thanks to Jim’s strict regime

that required each of us to produce 2,000 words per day, no

matter which day. 600 pages were very close to our

estimate – but they only covered less than half the topics

we wanted to discuss. So in order to preserve the

investment, we had to plan for a second hideaway, which

took place one year later in Bolinas (north of San

Francisco), again from February to April. At the end of this

period, we had about 1,000 pages of text, plus a number of

lessons learned

- We would not be able to cover all the material that

was contained in the foils of the course.

- We would still have to do a lot of work in order to get

the book to the printer (glossary, index, and proof

reading).

- Writing a book is hard work; we would never do it

again.

So Robert Burns was right indeed: The best laid plans …

3. ORGANIZING THE MATERIAL

In the years between the Berlin course and the time of

finishing the book, technologies related to transaction

processing, distributed computing, parallel databases, etc

developed at a rapid pace. There is by far not enough space

to list them all, but I will mention some that had a major

influence on the way the book was structured.

First, transaction technology for distributed systems started

to be used seriously on non-proprietary operating system

platforms, i.e. Unix [4]. This partly was the result of

transferring research results from academia into real

products via start-ups. A particularly notable example for

this was Transarc’s Encina-system [7], which was the result

of a multi-year research effort at CMU, led by Alfred

Spector. Since Jim had been discussing with this group on a

regular basis, he had very detailed insight into both the

architecture and the implementation of the system.

Second, the ideas for making transactions a fundamental

mechanism for reliable execution at all levels of a system

rather than just use it for database applications were being

transformed into real systems. The most advanced example

in that category was Tandem’s TMF [6], which

demonstrated the use of transactions in the operating

system and featured real transactional RPCs, among other

interesting things. Of course, Jim was particularly familiar

with that system, so we often used it as a reference when

discussing how the elements of a “good” transaction

processing system (for teaching purposes) should play

together.

Third, C. Mohan of IBM had started to systematize and

clarify many techniques for implementing transactional

execution that had been around in various systems for

many years and present them in a coherent framework. This

resulted in a famous series of papers on the ARIES design

[5], which had a significant influence on how we presented

recovery mechanisms and methods for synchronization on

B-tree structures, among other things.

In several companies and many research labs people were

working on new synchronization protocols, disaster

There was yet another lesson, having to do with the deeply

rooted connection between transaction processing and the level

of precipitation in the area one writes about it - but that is

beyond the scope of this short article.

SIGMOD Record, June 2008 (Vol. 37, No. 2)

Is There Life Outside Transactions?

Writing the Transaction Processing Book

Andreas Reuter

European Media Laboratory

Schlosswolfsbrunnenweg 33

D-69118 Heidelberg

+49-6221-533200

reuter@eml.org

ABSTRACT

In this article I will reflect on the writing of “Transaction

Processing – Concepts and Techniques” [1], which

appeared at Morgan Kaufmann Publishers in 1992. The

process of writing had many aspects of a typical software

project: In the end, the book was more than twice as thick

as we had planned, it covered only ¾ of the material that

we wanted to cover, and completing it took much longer

than we had anticipated. Nevertheless, it was a moderate

success and served as a basic reference for many

developers in the industry for at least 10 years after its

publication. It was translated to Chinese and Japanese, and

occasionally one still finds references to it – despite the fact

that (apart from simple bug fixes) there has been no

technical update of the material, and the book deals with

“outdated” subjects like transaction processing and

client/server architectures.

Categories and Subject Descriptors

K.2 [History of Computing]: People

General Terms

Performance, Design, Reliability, Standardization.

Keywords

Transaction processing, fault tolerance, software

architecture.

1. INTRODUCTION

In 1986 Jim had signed a contract with a seminar organizer

for teaching a one-week course on transaction processing in

spring 1987. The course was to take place in Berlin, and

because he did not want to teach the full five-day load all

by himself, he invited me to share some of it, assuming that

university professors have most – if not all – of the course

material ready for delivery on short notice. For the

following eight months we worked on preparing the slides.

The “plan” guiding the process was a list of chapter

headings that each of us would work on, with each chapter

representing a 90 min lecture. There was very little

communication along the way: A phone call now and then,

but no exchange of drafts or anything like that. Exchanging

emails between a university and a (closed) company

environment – Tandem in our case – was not something

you could simply do, and slides were either drawn by hand

or printed and then copied onto transparencies

. Two weeks

before the course started we sent a complete set of paper

copies of our 980 foils to the organizer, who turned them

into handouts for the participants. Delivering them all

would have been quite a challenge, but as soon as we found

out that the participants were very knowledgeable

developers with good ideas and tricky questions (I

remember Bill Laing and Paul Shrager being among them),

Jim decided to change the course structure completely.

Instead of following the table of contents in the handouts

we focused on what the participants were interested in and

skipped those parts that nobody wanted to hear about. For

example, we discussed a design where (for reliability) the

requests are sent to two completely independent systems

and executed on both. The question was, under which

conditions one could guarantee that the transactions are

serialized in the same order on both systems. So it was a

very lively, highly interactive course. I do not know how

much the participants took home, but the two presenters

definitely learned a lot.

2. ROBERT BURNS WAS RIGHT

When we reviewed the course, Jim observed that the

students’ questions had produced something that our initial

organization of the material had not suggested: A systems-

oriented perspective on transaction processing. So instead

of describing TP technology in isolation, and then

describing databases, networking, programming issues, etc.

we presented transactional concepts as some kind of

unifying framework for all layers of a system, from the

operating system all the way up to the applications and the

user interfaces. Jim went on saying that there were no

textbooks taking that integrative approach, and that it

should not be too hard to turn our 980 foils into text. We

estimated that on average two slides would transform into

one page of prose (including tables and figures), so that we

would have to write ca. 500 pages – 250 pages per person.

Doing this within a year seemed quite reasonable at the

In 1987 Microsoft bought the company that had produced the

precursor to Powerpoint, but it took a couple of years for that to

impact our teaching habits.

SIGMOD Record, June 2008 (Vol. 37, No. 2)

time, given that we had most of the material already. This

is how the whole thing started.

Before we actually tried to implement it, the initial plan

seemed to make perfect sense: The major portion of the

work had already been completed by putting all the

technical material on the slides – or so we thought. We

would only have to convert bullets into sentences, redo

some of the figures, add a chapter drawing all the details

into the grand picture of transaction-oriented systems,

compile a list of references – and be done! It was the kind

of plan that everybody will enthusiastically agree to at the

end of a meeting, so they can get on with their real work. In

our case it was the review meeting after the course, and

neither Jim nor I had a clear idea of how to implement it

after we got back from Berlin. However, with the best of

intentions we agreed on producing text from the slides

some time soon.

With no deadline at all and many other things to do, we

made very little progress in turning the foils into prose – in

fact, we did not make any progress at all. I used the

material for a variety of courses I taught at the university,

extending and changing it as new algorithms, new systems

etc. became available. Jim did the same, teaching

transaction processing at Stanford, but we still were just

using and updating the slides; no prose was being produced

as a result of the teaching activities. The only new type of

content that proved useful when – much later – we actually

wrote the book was a rapidly growing number of problems

and exercises related to the various topics that were

covered in the foils. Those problems were specifically

created for the university courses; they had not been part of

the Berlin seminar.

That was the situation in 1987, and it did not change in

1988, or in 1989. In the fall of 1989 we discussed the

project and found that the original plan had been a failure.

It was obvious that if we wanted to get anything written, we

would have to hide in some remote, quiet and pleasant

place, equipped with PCs, printer, toner, with easy access

to good food, and spend all our time typing – well, most of

it. We figured that three months should be enough to

produce a complete first draft of the book, the polishing of

which could be done later, when we were back in our

normal habitats. After some lengthy and careful

deliberation it was decided to rent a house in a small village

in Tuscany named Ripa (near Carrara) and spend February

through April of 1990 there.

This time we got it partially right: At the end of April we

had about 600 pages of text, thanks to Jim’s strict regime

that required each of us to produce 2,000 words per day, no

matter which day. 600 pages were very close to our

estimate – but they only covered less than half the topics

we wanted to discuss. So in order to preserve the

investment, we had to plan for a second hideaway, which

took place one year later in Bolinas (north of San

Francisco), again from February to April. At the end of this

period, we had about 1,000 pages of text, plus a number of

lessons learned

- We would not be able to cover all the material that

was contained in the foils of the course.

- We would still have to do a lot of work in order to get

the book to the printer (glossary, index, and proof

reading).

- Writing a book is hard work; we would never do it

again.

So Robert Burns was right indeed: The best laid plans …

3. ORGANIZING THE MATERIAL

In the years between the Berlin course and the time of

finishing the book, technologies related to transaction

processing, distributed computing, parallel databases, etc

developed at a rapid pace. There is by far not enough space

to list them all, but I will mention some that had a major

influence on the way the book was structured.

First, transaction technology for distributed systems started

to be used seriously on non-proprietary operating system

platforms, i.e. Unix [4]. This partly was the result of

transferring research results from academia into real

products via start-ups. A particularly notable example for

this was Transarc’s Encina-system [7], which was the result

of a multi-year research effort at CMU, led by Alfred

Spector. Since Jim had been discussing with this group on a

regular basis, he had very detailed insight into both the

architecture and the implementation of the system.

Second, the ideas for making transactions a fundamental

mechanism for reliable execution at all levels of a system

rather than just use it for database applications were being

transformed into real systems. The most advanced example

in that category was Tandem’s TMF [6], which

demonstrated the use of transactions in the operating

system and featured real transactional RPCs, among other

interesting things. Of course, Jim was particularly familiar

with that system, so we often used it as a reference when

discussing how the elements of a “good” transaction

processing system (for teaching purposes) should play

together.

Third, C. Mohan of IBM had started to systematize and

clarify many techniques for implementing transactional

execution that had been around in various systems for

many years and present them in a coherent framework. This

resulted in a famous series of papers on the ARIES design

[5], which had a significant influence on how we presented

recovery mechanisms and methods for synchronization on

B-tree structures, among other things.

In several companies and many research labs people were

working on new synchronization protocols, disaster

There was yet another lesson, having to do with the deeply

rooted connection between transaction processing and the level

of precipitation in the area one writes about it - but that is

beyond the scope of this short article.

SIGMOD Record, June 2008 (Vol. 37, No. 2)

of 5

5墨值下载

数据库简史

文档被以下合辑收录

数据库简史（共33篇）

关于我的新书《数据库简史》中引用的一些资料，统一辑录在这里。

关注

文档被以下合辑收录

评论