An Analysis of Introductory Programming Courses at UK Universities

Context: In the context of exploring the art, science and engineering of programming, the question of which programming languages should be taught first has been fiercely debated since computer science teaching started in universities. Failure to grasp programming readily almost certainly implies failure to progress in computer science. Inquiry: What first programming languages are being taught? There have been regular national-scale surveys in Australia and New Zealand, with the only US survey reporting on a small subset of universities. This the first such national survey of universities in the UK. Approach: We report the results of the first survey of introductory programming courses (N=80) taught at UK universities as part of their first year computer science (or related) degree programmes, conducted in the first half of 2016. We report on student numbers, programming paradigm, programming languages and environment/tools used, as well as the underpinning rationale for these choices. Knowledge: The results in this first UK survey indicate a dominance of Java at a time when universities are still generally teaching students who are new to programming (and computer science), despite the fact that Python is perceived, by the same respondents, to be both easier to teach as well as to learn. Grounding: We compare the results of this survey with a related survey conducted since 2010 (as well as earlier surveys from 2001 and 2003) in Australia and New Zealand. Importance: This survey provides a starting point for valuable pedagogic baseline data for the analysis of the art, science and engineering of programming, in the context of substantial computer science curriculum reform in UK schools, as well as increasing scrutiny of teaching excellence and graduate employability for UK universities.


Introduction
For many years -and increasingly at all levels of compulsory and post-compulsory education -the choice of programming language to introduce the "art" [ ], "science" [ ] and "discipline" [ ] of computer programming via key programming principles, constructs, syntax and semantics has been regularly revisited. Even in the context of what are perceived to be the most challenging introductory topics in computer science degrees, numerous key themes across programming frequently appear [ ].
So what makes a good first programming language? Perhaps more importantly, how are we defining "good"? The issues surrounding choosing a first language [ , ] -and a search of the ACM Digital Library identified a number of papers of the form "X as a first programming language", going as far back as the s [ ] -appear to be legion, especially with wider discussions of precisely what we aim to achieve from teaching programming [ , , , , ]; from the potential impact on students' grades and attainment [ , , , , ], addressing gender and diversity issues [ , ], to a renewed focus on developing transferable computational thinking and problem solving skills [ , , ]. There is a belief that programming -as opposed to, say analysis of algorithms, a closely related theoretical skill -is fundamentally a craft that needs immersion and practice [ , ]. It appears that decades of research on the teaching of introductory programming has had limited effect on classroom practice [ ]; although relevant research exists across several disciplines including education and cognitive science, disciplinary differences have often made this material inaccessible to many computing educators. Furthermore, computer science instructors have not had access to comprehensive surveys of research in this area [ , ].
However, in Australia and New Zealand there have been longitudinal data collections [ , , ] surveying the teaching of introductory programming courses in universities. Surprisingly, such surveys have not been conducted elsewhere on this scale (in the USA, [ ] only covers the "top " universities), and this paper reports the findings from running the first such similar survey in the UK.
We remind the reader that the UK consists of four nations with an overall population of . million: England ( . million), Scotland ( . million), Wales ( . million) and Northern Ireland ( . million). In , Scotland and Wales held referendums which determined in both cases the desire for increased self-government (along with Northern Ireland and the Good Friday Agreement), creating national assemblies to which a variety of powers -in particular, education -were devolved from the UK Parliament. Thus, we now have an educational policy ecosystem in the UK that was historically ruled by one parliament but now comprising three devolved assemblies responsible for four separate education systems.
In the context of increasing international focus on curriculum and qualification reform to support computer science education and digital skills in schools, the four education systems of the UK have proposed and implemented a variety of changes [ , , , ] impact on the delivery of their undergraduate programmes, with the diversity of the educational background of applicants likely to increase before it narrows: it is certainly possible now for prospective students to have anywhere from zero to four or more years experience (and potentially two school qualifications) in computer science with some knowledge of programming.
Since , there has been increasing scrutiny of the quality of teaching in UK universities, partly linked to the current levels -and potential future increases -of tuition fees (generally paid by the student through government-supported loans), as well as relative levels of graduate employability and the perceived value of professional body accreditation by industry. In February , the UK (but in this respect only responsible for England) Department of Business, Innovation & Skills initiated independent reviews of science, technology, engineering and mathematics (STEM) degree accreditation and graduate employability,¹ with a specific focus -the Shadbolt review [ ] -on computer science degree accreditation and graduate employability, reporting back in May . Alongside a number of recommendations to address the apparently relatively high unemployment rates of computer sciences graduates, particular on the quality of data, course types, gender and demographics, the Shadbolt review split generalist universities in England into three bands ("low", "medium", "high"), based on the average (across all subjects) entrance tariff of incoming undergraduates (as defined by the national UCAS Tariff²); we have followed the same tariff banding during our analysis of the English results, so our data should allow comparisons.
Thus, in this evolving environment of new policies and curricula, as well as the emerging demands of innovative pedagogies and high-quality learning and teaching for computer science degree programmes, we present the findings from the first national scale survey of introductory programming languages at UK universities, providing a baseline for deeper analysis of the art, science and engineering of programming. Through this first UK-wide survey of universities, we identify and analyse trends in student numbers, programming paradigm, programming languages and environment/tools used, as well as the underpinning rationale for their selection.

Methodology . Recruitment of Participants
To recruit for the survey, a general call for participants was sent out to the Council of Professors and Heads of Computing (CPHC) membership; CPHC³ is the representative body for university computer science departments in the UK, with nearly members at over institutions. The survey was hosted online from mid-May until the end of https://www.gov.uk/government/collections/graduate-employment-and-accreditation-instem-independent-reviews https://www.ucas.com/advisers/guides-and-resources/tarihttps://cphc.ac.uk -

Figure
The number of responding universities per Nation/Tariff Group June , with the invitation asking for the survey to be passed on to and completed by the most appropriate person in that institution. Due to the recruitment method, there were a number of duplicate responses from certain departments, and these were reconciled by direct enquiry.
The questions used in the survey were generously provided by the authors of the Australia and New Zealand survey [ ], so as to allow direct comparison with the results of this survey. Where possible, questions were left unchanged, although a small minority were edited to reflect the UK target audience. As defined in the Aus/NZ survey, the terminology "course" was used for "the basic unit of study that is completed by students towards a degree, usually studied over a period of a semester or session, in conjunction with other units of study".
The first section of the survey asked about the programming language(s) in use, the reasons for their choice, and their perceived difficulty and usefulness. Then, questions regarding the use of environments or development tools; which ones were used, the reasons for their choice and the perceived difficulty. General questions about paradigm, instructor experience and external delivery were asked, along with questions regarding students receiving unauthorised assistance, and the resources provided to students. Finally, participants were asked to identify their top three aims when teaching introductory programming, and were also allowed to provide further comments.
In the Aus/NZ survey, participants were asked to rank the importance of the reasons for choosing a programming language, environment or tool. Due to technical limitations in the online survey tool used, it was not possible to do so in this survey, so "what programming language(s) are in use?" and a small number of feeder questions to allow the survey to function correctly.

. Universities and Courses
Upon completion of the survey, instructors had, at least, started the survey; of these dropped out before answering the mandatory questions, and a further were duplicates. Therefore, the results presented here are drawn from the responses of instructors from at least institutions. Some participants did not answer all questions and thus the response rate varies by question.
Excluding the Open University's students, the participants in the survey represented students, with a mean of (but a standard deviation of ). Looking at Figure we see good response rates, apart from the specialist higher education institutions (most of whom do not teach computing) and the "low tariff" English ones. Fewer of these teach computing; this factor alone explains the response rate. In Northern Ireland, we had responses from the two universities, but not the university colleges, which are historically initial teacher education colleges. -

Figure
Reasons given for choosing a programming language by percentage for: all languages; Java; and Python

. Languages
A primary focus of this survey was to identify the programming languages in use in introductory programming courses. Participants were asked to select languages from a list of programming languages and also had the option to choose "Other" and specify a language not included in the list. The majority of courses surveyed ( out of , . %) use only one programming language, with using two (and only three and one institutions using three and four languages respectively). From the courses, the total number of language instances is , as some courses use more than one language to teach introductory programming.
Of the languages provided, were selected at least once. The relative popularity of languages is shown in Figure , where the prevalence is given by the percentage of a language over all language instances ( total), and weighted by student numbers ( total) per language instance. The programming languages that were not selected at all were: Actionscript, Ada, Delphi, Eiffel, Fortran, jBase, Lisp, Ruby and Visual Basic.
The relative popularity of languages is the immediate major difference with the Aus/NZ survey; their survey showed a dead heat ( . % of language instances) between Java and Python, with Python winning ( . % to . %) when weighted by the number of students enrolled on the course. Our findings in Figure  -An Analysis of Introductory Programming Courses at UK Universities use in . % of courses and makes up . % of language instances. The C family (C, C++ and C#)⁴ together is in use in . % of introductory programming courses, and scores . % of language instances and . % by students. Figure also shows the effect of student-number weighting but we have excluded the Open University from this weighting, as its students learning Python (and Sense, a variant of Scratch) would have distorted the comparison.
For each language selected, participants were asked to give the reasons for choosing that language for the introductory programming course. Figure shows the frequency of these reasons for all languages grouped together and for Java and Python individually. When the reasons given are combined for all languages, three reasons tie for first place: "relevance to industry"; "object-oriented language"; and "availability and cost to students", all chosen by . % of participants who answered this question.
Looked at individually, the most popular reason given for choosing Java is "objectoriented language" at . %, while Python scores highest on "pedagogical benefits", at . %. This may explain the popularity of Java: Java scores higher on "relevance to industry" and, perhaps somewhat surprisingly, much higher on "object-oriented language" than Python, which only scores . %. Figure breaks down the choice of language by nation and tariff group. It is noticeable that the three English tariff groups differ significantly, with Python outnumbering Java in the low tariff universities, and C being almost exclusively in the high tariff universities. Figure gives the instructors' views on languages. It is noteworthy that Java is among the most difficult, and not among the pedagogically most useful.
For each language chosen, instructors were asked whether the language was used: for the whole of the first programming course; for the first part of the first programming course, followed by another; after another language in the first programming course. Of language instances, the majority ( %) are used for the whole of the introductory programming course, . % of language instances are used in the first part of a course and . % of language instances are used after another programming language; results are displayed in Figure . .

Paradigm Taught
Instructors were asked which paradigm was being taught in their introductory programming course, regardless of what is traditionally thought to apply to the language(s) in use. This question, understandably, caused some dissatisfaction in the comments section, with many participants noting that more than one paradigm is taught in their course. Although this was to be expected, we wanted to be able to directly compare our results to the Aus/NZ survey, and so did not alter the question. The most popular paradigm is object-oriented with % (N = 40, %) One referee queried whether C# counts as "C family", describing it as "much closer to Java". One can find apparently authoritative statements in both camps from the language designers of Java and C#. Further analysis of the four C# instances shows four different patterns: C# only; a wide range of languages; C++ followed by C# and Java followed by C#.

Figure
Tool or environment popularity by percentage of courses and students followed by procedural (N = 27, . %) and functional (N = 7, . %); logical was also offered as a choice but was not selected. The results of the previous question were used to analyse -see Figure -the prevalence of paradigms across nations and tariff score groups. Caution must be applied when interpreting these results, as participants could only choose one paradigm, even though more may be in use.
In the same way as above, the languages chosen were analysed -see Figure -with regard to the main paradigm in use. Again, caution must be applied, as for a given course, only one paradigm is chosen, even though more than one language and/or paradigm may be in use. This explains the respondents who used C, but stated that object-oriented was the main paradigm, for example. More surprising is the fact that Python was almost exclusively viewed as procedural.

. Instructor Experience
Participants were asked: "How many years have you been involved in teaching of introductory programming?". The results, shown in Table , indicate that of the survey participants, the average was between -years.

Figure
Reasons given for choosing a tool or environment by percentage for: all tools and environments; BlueJ; and Eclipse

. IDEs and Tools
Participants in the survey were asked if they encouraged students in the first programming course to use environments and/or tools beyond simple text editors and command line compilers. The majority of participants of this question ( . % of instructors) responded that they did encourage tools. Of the instructors that did select a tool/IDE, the majority ( . %) use only one, with . % and . % using two and three respectively; very few ( . %) used four or more, with one respondent using eight.
The survey asked participants to select the tools and IDEs in use in their introductory programming course out of a list of , with the option to specify "Other". Of the provided, were chosen at least once. The relative popularity of IDEs and tools is shown in Figure . The most popular tool/IDE in the survey was Eclipse, reported in . % of courses and scoring . % of tool/IDE instances, and . % when weighted by students. Following this is "No Tool/IDE", which accounts for . % of courses. The second most popular tool/IDE is BlueJ, which was reported in . % of courses and scored . % of tool/IDE instances, and . % when weighted by students. Participants were also asked why each tool/IDE was chosen for their course, and asked to select from a list of reasons. The results of this are give in Figure , for all tools and IDEs grouped together, and for the two most popular choices, Eclipse and BlueJ. The tools and IDEs not selected at all were: AdaCore, Alice, App, Browser, Greenfoot, Jeroo, Jython, KTechLab, MySQL, Pelles, Quincy, Wing and Xcode.

Figure
For each tool or environment, whether it is used: for an initial part of the first programming course; throughout the whole of the first programming course; in any other course in the degree

. . Reuse of Tool/IDE
Instructors were also asked whether the tool/IDE was used for an initial part of the first programming course or throughout the whole of the course; and whether it was used in any other course in the degree (Figure ).

. . Di culty of Tool/IDE
In addition to this, instructors were asked to rate how difficult they found the tool/IDE on a Likert scale from "Extremely Easy" ( ) to "Extremely Difficult" ( ), and also how difficult they believed the students found the tool/IDE, shown in Figure . We note that, while Eclipse is the most popular tool by some way, it is also deemed to be most difficult. This, apparently perverse, practice might be explained by the extent of re-use of Eclipse in other courses.

. . External Delivery
Participants were asked "Do you offer external delivery of your course? (i.e. do you have options for your course where students are not required to attend regular lectures, workshops, labs or tutorials?)". The responses to this question were overwhelmingly in the negative; / ( . %) answered "No".

Figure
The median difficulty rating of tool for the instructor and students to use, where is 'extremely easy' and is 'extremely difficult'

Figure
Steps taken to determine whether students have received unauthorised assistance on assignments

. . Resources Provided to Students
The questionnaire asked about the resources in terms of examples, books etc. provided to students. The results are rather similar to the Aus/NZ survey [ , Figure ] and are displayed in Figure . The most popular resources selected were: "lecture slides or notes provided by the lecturer" in first place, "worked examples of programming problems/solutions" in second, and third place was shared by "textbook is specified" and "discussion boards/forums".

. . Unauthorised Assistance
The vast majority of instructors surveyed ( . %) do consider the possibility that students or groups of students may be receiving unauthorised assistance (e.g. from other students in the class, from people outside the class, or via the internet) when doing assignments. When asked how concerned they were about this possibility, answered "not concerned", answered "somewhat concerned" and reported "very concerned".
We also asked participants: "What steps do you take to try to determine whether students have received unauthorised assistance on assignments?". The details are displayed in Figure  and range from "notice unlikely similarities" ( . % of the instructors who responded to this question) to "interview some students/groups at random", selected by only five instructors.

. Aims of an Introductory Programming Course
The Aus/NZ survey asked their respondents for the aims of their introductory programming course. They, and we, asked for (up to) three aims. The authors then attempted to classify the free-text answers into the same categorisation as [ ] used. While it is trivial to map the written aim "Thinking algorithmically" to "Algorithmic thinking" [ ] and so on, many were not so clear: for example, we mapped "To learn a specific language" to "Syntax/writing basic code". There were also a class of aims, such as "Establish professional software development practices", that seemed coherent, but did not map clearly to the [ ] aims; these we have categorised as "Software Engineering". Results of this question are shown in Figure .

. Comparison with Australasian Survey
Here we compare with the latest Australasian survey [ ]; we have already commented on the major difference in language choice, which colours many of the other comparisons. In fact, the UK's language choices seem more similar to Australasia's choices [ ] and [ , Table ] than even Australasia's choices. It is hard to know which comes first, but we also notice that our difficulty/utility data ( Figure ) is somewhat different from [ , Figures / ]. Another difference in the tools/environments used is demonstrated by Figure  versus [ ]'s Figure . There, "None" and "Other" were the top two categories, with IDLE, at %, the most popular named product. In the UK, "None" is second, "Other" is sixth and IDLE eighth. Eclipse, the UK favourite, was an "also ran" in [ ].

. The UK Context
As presented in Section . , our findings show that Java is the most popular introductory programming language in UK universities, more than twice as popular as Python in second place; the C family of languages (C, C++ and C#) together is in use in nearly a third of introductory programming courses. We were surprised by the viewpoint expressed of Python, as a multi-paradigm language, as being largely procedural⁵; from the authors' experiences, the dominance of Java has been a trend for the past ten years, but we would expect to see a steady increase in Python due to influences from the changes to school curricula in the UK [ ].
We note that from a smaller survey conducted in July , Python is the most popular language for teaching introductory computer science courses at top-ranked US university departments; specifically, eight of the top CS departments ( %), and of the top ( %), teach Python in introductory CS or CS courses [ ]. This together with [ ] and Figure might make one question the UK's domination by Java, although longstanding industry popularity as measured by community indices may still be a significant determining factor [ , ].
From a UK education policy perspective, a new national Teaching Excellence Framework has been proposed, with a core ambition to "to raise the quality and status of teaching in higher education institutions"; excellence is to be measured through a series of proxy metrics that include student satisfaction, retention and graduate employability.⁶ There have been significant sector concerns about the aims of the framework -as well as the statistical rigour of the metrics -more so in the context of it being used for benchmarking⁷ "teaching excellence" (particularly as the TEF will not yet be conducted at the individual subject level, but at the institutional level), as well as deciding whether institutions are allowed to raise tuition fees in the future. It remains to be seen how this will affect undergraduate computer science degree curricula in UK institutions going forward, especially if there is renewed demand for We can only speculate why this is; one reason could be the nature of many of the texts available: see [ ], and for example a popular freely-available Python text [ ] which the third author has used while teaching teachers introduces classes only in chapters -(of in total).
http://www.hefce.ac.uk/lt/tef/ Many UK newspapers produce "University League Tables", all based on much the same published data. The new Teaching Excellence Framework will grade universities as bronze/silver/gold, and it seems inevitable that the newspaper league tables will use these in their league tables.
meeting the immediate (but potentially transient) demands of the IT industry with specific tools, languages and environments, as well as reformed professional body accreditation as per the Shadbolt review [ ]. The UK's Higher Education Academy -the national body which champions teaching quality -has previously supported initiatives for improving learning and teaching in computer science, including innovative pedagogies for programming [ , ], but we have not yet seen the necessary development of sustainable discipline-specific communities of practice, both at the local and national level, to capture and share best practice.⁸

Future Work
This national survey provides valuable context for better understanding of the role and effectiveness of programming education in UK universities. Furthermore, how this impacts more broadly across the education pipeline: through significant curriculum reform, as well as scrutiny of the effectiveness of pedagogies for teaching principles of programming and software engineering (in essence: software carpentry, providing the knowledge, skills and understanding to create useful and usable software for a variety of domains). Moving forward will require an mixed economy of rigorous pedagogical research, as well as the application of personal experiences of languages, tools, environments, models and styles. Only through this blend of the art, science and engineering of programming will we see significant steps towards improving programming (and thus computer science) education in the UK. Those data created during this research project that do not infringe the anonymity of the respondents are openly available from the University of Bath data archive at http://doi.org/ .
/BATH-. However, a new initiative announced by the Royal Society in is aiming to addresses this in UK schools: https://royalsociety.org/topics-policy/projects/computing-education/ -