The Need for Richer Refactoring Usage Data

Exceptional Cases

Modern IDEs support many refactorings.

Methods of Invocation

Manual and Automated Refactorings

Awareness

Previews

Configuration

Why don't developers take full advantage of refactorings tools?

There has been active research on technical aspects and a growing interest in usability aspects of refactorings.

The user interfaces of refactoring tools have changed little.

The Need for Richer Refactoring Usage Data

We need richer usage data and interviews to understand the impact of error messages on the use of automated refactorings.

Eclipse Refactoring Tool

We need richer usage data and interviews to understand why programmers sometimes prefer to perform a refactoring manually.

Smalltalk Refactoring Browser

We need richer usage data and interviews to answer questions related to awareness.

We need richer usage data and interviews to understand the impact of previews on the use of automated refactorings.

We need richer usage data and interviews to understand the impact of configurations on the use of automated refactorings.

Existing automated refactorings are underused.

We combine quantitative and qualitative methods to study the use of automated refactorings.

OOPSLA'11

We need richer usage data to find the difficulties in invoking automated refactorings.

We have submitted a paper about our results, which is under review.

ICSE'011

ICSE'11

CodingSpectator and CodingTracker capture richer data that make it possible to answer many open research questions.

Mohsen Vakilian, Nicholas Chen, Stas Negara, Balaji Ambresh Rajkumar, Roshanak Zilouchian Moghaddam, Ralph E. Johnson

ICSE'11

ASE'11

What are the existing refactoring data sets?

OOPSLA'10

ECOOP'10

Cancellations and Undos

{mvakili2, nchen, snegara2, rajkuma1, rzilouc2, rjohnson}@illinois.edu

CodingSpectator

E. Murphy-Hill, C. Parnin, and A. P. Black, “How We Refactor, and How We Know It,” IEEE Transactions on Software Engineering, no. PrePrints, 2011

• Manual and Automated Refactorings
• Awareness
• Methods of Invocation
• Configuration
• Previews
• Exceptional Cases
• Cancellations and Undos

CodingTracker

ICSE'09

• What automated refactorings present error messages more frequently?
• How do error messages affect the developer’s future uses of the tool?

Existing refactoring data do not reveal many problems with refactoring tools.

ICSE'09

Our Preliminary Results

UDC has collected data from thousands of users during the past 3 years.

• How often do programmers preview automated refactorings?
• What refactorings are programmers more likely to preview?

Mylyn Monitor was the first usage data collector for Eclipse.

FSE'09

ASE'09

OOPSLA'08

• What refactorings are more difficult to invoke? What refactorings programmers fail to invoke because of wrong input selections?

• How long do developers spend on configuring various automated refactorings?
• Are long configuration times indicators of usability problems?

ICSE'08

We collected data from 1448 hours of programming of 26 programmers and interviewed nine participants.

University of Illinois at Urbana-Champaign

• How common are simple and complex automated refactorings?
• How often are different refactorings performed with and without tools?

VL/HCC'11

ICSE'07

We need richer usage data and interviews to understand if cancellations and undos of refactorings are indicators of usability problems.

FSE'07

SOFTVIZ'10

• What refactorings do developers frequently underuse because of their unawareness?
• How do developers become aware of certain automated refactorings?

ICSE'05

ICSE'09

D. Roberts, J. Brant, and R. Johnson, “A Refactoring Tool for Smalltalk,” Theory and Practice of Object Systems, 1997.

PLATEAU 2011

G. C. Murphy, M. Kersten, and L. Findlater. How Are Java Software Developers Using the Eclipse IDE? IEEE Software, 23:76–83, July 2006.

ICSE'08

ECOOP'05

Manual and Automated Refactorings

ASWEC'07

OOPSLA'00

Interview

Technical

Usability

Refactorings may be recovered from version control histories.

• 80% of the automated refactorings that our participants performed affected at most six lines.
• 84% of the automated refactorings that our participants performed affected only one file.

Eclipse refactoring histories are more detailed.

Richer Usage Data

Awareness

• Do the error messages of automated refactorings influence developers to cancel or undo their refactorings?

• On average, our interviewees were unaware of the existence of more than nine of the 26 automated refactorings of Eclipse.
• On average, our interviewees did not know the goals of more than eight automated refactorings.

http://www.flickr.com/photos/visual_dichotomy/3623619145/

Configuration

• 80% of the automated refactorings that our participants performed required at most eight seconds of configuration.

http://codingspectator.cs.illinois.edu

Previews

• Our participants previewed only 1% of the automated refactorings they invoked.

Exceptional Cases

• Our participants chose to continue 66% of the refactorings that had reported some errors.