03 【B-可持续性】Generative AI for Software Practitioners .pdf

Zixin Huo

141

9页

0次

2023-10-14

免费下载

GENERATIVE ARTIFICIAL INTEL-

LIGENCE (AI) has the potential to

change the software profession more

than any other recent technology.

Bill Gates sees it as the biggest move

forward since the invention of the

Internet. It can improve software

productivity in several ways, such

as automating repetitive tasks (e.g.,

testing or requirements traceability),

improve software quality by creat-

ing test suites from requirements,

and automate workflows by rout-

ing work products to the next suit-

able step in a production pipeline.

At the same time, generative AI cre-

ates fully new risks because it is nei-

ther deterministic nor explainable.

IPR and cybersecurity are prominent

examples that limit usage in profes-

sional software engineering.

Digital Object Identifier 10.1109/MS.2023.3265877

Date of current version: 14 July 2023

Generative AI for

Software Practitioners

Christof Ebert and Panos Louridas

Editor: Christof Ebert

Vector Consulting Services

christof.ebert@vector.com

SOFTWARE

TECHNOLOGY

©SHUTTERSTOCK.COM/DEEMERWHA STUDIO

From the Editor

Generative artificial intelligence (AI) tools, such as Bard, ChatGPT, and CoPilot, have

rapidly gained widespread usage. They also have the potential to boost software en-

gineering productivity. In this article, we elaborate technologies and usage of genera-

tive AI in the software industry. We address questions, such as: How does generative

AI improve software productivity? How to connect generative AI to software develop-

ment, and what are the risks? Which technologies have what sorts of benefits? Practi-

tioner guidance and case studies are shared from our industry context. I look forward

to hearing from you about this column and the technologies that matter most for your

work.—

Christof Ebert

Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on October 11,2023 at 13:25:56 UTC from IEEE Xplore. Restrictions apply.

SOFTWARE TECHNOLOGY

JULY/AUGUST 2023 | IEEE SOFTWARE

Generative AI Technologies

Generative AI has been around for

many years. With no means to prove

validity, researchers hesitated to bring

such technology to the mass market

of rather naïve data citizens. As we

have observed many times in recent

IT history, the perceived gold rush

makes people close their eyes to ob-

vious risks. Even tools designed for

good will eventually have devastating

consequences. When ChatGPT was

finally released to a wide public audi-

ence in 2022, the AI arms race started

at a speed never seen before. It took

just two months for ChatGPT to reach

100 million users. Figure 1 shows this

fast evolution for different technolo-

gies spanning a mere 100 years of

recent human history. A technology

like the wheel even took thousands of

years to reach 100 million users.

For every developer, turning to

StackOverflow or Google has been a

natural part of the job for years now.

Condemning the “not invented here

syndrome” to the dustbin, our first

reaction when in doubt how to code

something has been to look it up on

the Internet. Search engines have be-

come better at indexing code reposito-

ries, myriads of which exist online, and

community advice sites, such as Stack-

Overflow, provide reasoned solutions

and valuable commentary on user

questions. What is common in search

engines and question-and-answer web-

sites is that you can look up informa-

tion that has already been stored there.

Generative AI is different. As the

name suggests, it can synthesize—or

generate—the answers to the ques-

tions you pose. Instead of trawling a

prefabricated answer as classic search

engines are doing, it will create an

answer for you. The answer is based

on vast amounts of data on which

it has been trained, such as those ar-

chived and indexed by search engines.

To provide meaningful answers, gen-

erative AI undergoes further train-

ing based on human feedback. Many

human trainers pose questions and

provide feedback on the generated an-

swers, rewarding good answers and

punishing unsatisfying ones. This kind

of reinforcement learning guides the

system toward providing more accu-

rate answers, while guarding against

harmful responses. This has led to

glimpses of a new way of working,

where the focus is on “prompt engi-

neering”: find the most appropriate

way to frame a question or a whole

dialogue. Generative AI does not work

with individual question and answers:

it maintains a context window, which

can be used to guide the AI in gener-

ating contextually relevant and well-

informed responses.

Underlying all this, generative AI

is powered by large language models

(LLMs). As the name again suggests,

these are large neural network models

that are trained on big language cor-

pora. Technically, they have a trans-

former architecture, which is based on

a mechanism called attention. The pub-

lication of the attention mechanism, by

Google researchers, must now rank

among the most influential papers in

computer science.

Two early LLMs

were the Bidirectional Encoder Repre-

sentations from Transformers (BERT),

developed by Google in 2018,

and

FIGURE 1. Time to reach 100 million users for different technologies in months after

initial deployment.

ChatGPT

TikTok

Internet

Mobile Phone

Car

Telephone

190

400

900

Time to Reach 100 Million Users

0 100 200 300 400 500 600 700 800 900 1,000

The basic idea is to use a large

language corpus to train a neural

network to learn the language, by

hiding part of the text and asking the

network to guess the missing parts.

Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on October 11,2023 at 13:25:56 UTC from IEEE Xplore. Restrictions apply.

SOFTWARE TECHNOLOGY

JULY/AUGUST 2023 | IEEE SOFTWARE

Generative AI Technologies

Generative AI has been around for

many years. With no means to prove

validity, researchers hesitated to bring

such technology to the mass market

of rather naïve data citizens. As we

have observed many times in recent

IT history, the perceived gold rush

makes people close their eyes to ob-

vious risks. Even tools designed for

good will eventually have devastating

consequences. When ChatGPT was

finally released to a wide public audi-

ence in 2022, the AI arms race started

at a speed never seen before. It took

just two months for ChatGPT to reach

100 million users. Figure 1 shows this

fast evolution for different technolo-

gies spanning a mere 100 years of

recent human history. A technology

like the wheel even took thousands of

years to reach 100 million users.

For every developer, turning to

StackOverflow or Google has been a

natural part of the job for years now.

Condemning the “not invented here

syndrome” to the dustbin, our first

reaction when in doubt how to code

something has been to look it up on

the Internet. Search engines have be-

come better at indexing code reposito-

ries, myriads of which exist online, and

community advice sites, such as Stack-

Overflow, provide reasoned solutions

and valuable commentary on user

questions. What is common in search

engines and question-and-answer web-

sites is that you can look up informa-

tion that has already been stored there.

Generative AI is different. As the

name suggests, it can synthesize—or

generate—the answers to the ques-

tions you pose. Instead of trawling a

prefabricated answer as classic search

engines are doing, it will create an

answer for you. The answer is based

on vast amounts of data on which

it has been trained, such as those ar-

chived and indexed by search engines.

To provide meaningful answers, gen-

erative AI undergoes further train-

ing based on human feedback. Many

human trainers pose questions and

provide feedback on the generated an-

swers, rewarding good answers and

punishing unsatisfying ones. This kind

of reinforcement learning guides the

system toward providing more accu-

rate answers, while guarding against

harmful responses. This has led to

glimpses of a new way of working,

where the focus is on “prompt engi-

neering”: find the most appropriate

way to frame a question or a whole

dialogue. Generative AI does not work

with individual question and answers:

it maintains a context window, which

can be used to guide the AI in gener-

ating contextually relevant and well-

informed responses.

Underlying all this, generative AI

is powered by large language models

(LLMs). As the name again suggests,

these are large neural network models

that are trained on big language cor-

pora. Technically, they have a trans-

former architecture, which is based on

a mechanism called attention. The pub-

lication of the attention mechanism, by

Google researchers, must now rank

among the most influential papers in

computer science.

Two early LLMs

were the Bidirectional Encoder Repre-

sentations from Transformers (BERT),

developed by Google in 2018,

and

FIGURE 1. Time to reach 100 million users for different technologies in months after

initial deployment.

ChatGPT

TikTok

Internet

Mobile Phone

Car

Telephone

190

400

900

Time to Reach 100 Million Users

0 100 200 300 400 500 600 700 800 900 1,000

The basic idea is to use a large

language corpus to train a neural

network to learn the language, by

hiding part of the text and asking the

network to guess the missing parts.

Authorized licensed use limited to: Shenzhen Institute of Advanced Technology CAS. Downloaded on October 11,2023 at 13:25:56 UTC from IEEE Xplore. Restrictions apply.

of 9

免费下载

ieee

关注

评论