Vincenti explains all of this very well in "What Engineers Know and How They Know It", with the specific example of the "Flying Quality" of aircraft. Many educationalists would no doubt deny the possibility of the establishment of any reliable metrics for learning, let alone ingenuity, but I disagree.
I think that any good engineer can reliably and reproducibly tell a good engineer from a poor one sufficiently well to control the process of engineering education. Let us call the property they are estimating in making this judgement ingenuity.
As a good engineer I can tell from their average level of ingenuity that either the candidates I have been sent to make into engineers have been poorly selected, or (if these are indeed the best candidates available), the pool of sufficiently ingenious candidates is small.
I am not sure that I can see any significant differences in ingenuity between candidates of different genders, sexualities, skin colours and so on. It would not matter if I could, as it would be illegal for me to discriminate between candidates on these grounds (unless of course I was discriminating "positively", which I would rather not do).
Much of the discussion of how to improve engineering education centres on increasing numbers, both in general and of "under-represented" groups. The view of who is under-represented is however highly politicized. The great under-representation of the children of the working classes in engineering education is not apparently not a problem. The lack of a 50:50 gender ratio is however thought to be a problem. This is an ideological position, not a rational one. It is a political judgement, whose truth or otherwise is based on the acceptance or rejection of certain set of values and associated axioms.
We might set aside the issue of whether these values and axioms are right, and ask if this approach is likely to work, and if there is any evidence that it is working. But first we would have to agree about what "working" means. This is a problem I have come across many times in professional engineering practice. What is needed to resolve it rationally is an agreed metric.
In my recent discussions with Kel and Peter from the RAE, it seems clear to me that the immediate aims of the RAE's activities in encouraging applications to engineering courses are to produce twice as many engineering graduates, but that their ultimate aim is to produce more good engineers.
There is an implicit assumption in these approaches that more applicants in general, and more female applicants in particular will automatically lead to more good engineers. But this does not follow, whichever way you read the data on applications to engineering courses.
My belief is that we already train too many engineers in general, and poor engineers in particular. Even Peter's own figures suggest that almost half of our graduates are not being employed as engineers, and I think this analysis underestimates the problem due to the poor quality of its source data. Most professional engineers (including myself) think that today's engineering graduates lack ingenuity, and consider many of them unemployable as engineers.
The RAE's argument is that because some engineering courses have to go to "clearing" to make their numbers, we are not oversupplied with candidates. This to my mind still leads us to the conclusion that we have too many places available if our aim is to produce good engineers. Going to clearing means that these courses are taking on second choice students. If we believe that A-levels are measuring something which correlates with ingenuity, this dropping of standards reduces our chances of making good engineers.
Engineers have a feeling for the correct level of analysis. I have previously explained why I think analysis at the STEM level is unhelpful. Now I would like to explain why I think analysis at the "Engineering " level is unhelpful. Let us take the example of Chemical Engineering and Civil Engineering courses.
Even though only half of Chem Eng graduates get jobs as engineers, there has been a massive expansion in numbers and tariffs on Chem Eng Courses, and several new courses are being accredited. This may well be something to do that Chem Eng is the highest paid branch of engineering in the UK, more well paid than medicine. Chem Eng courses are 27% female, and Biochem Eng courses even closer to gender parity.
Contrast this with the least well paid branch of UK engineering, Civil Engineering. Civ Eng can't fill its courses, and it can't get the girls. So Civil Engineering may have a problem - does Chem Eng have the same problem? I think not. In my opinion Chem Eng is probably exceeding, and certainly approaching the saturation point for willing and able female candidates.
So it seems to me that in a system where we charge students £36K for an entry level engineering qualification, they are voting with their feet for the courses most likely to give a good return on their investment. Anyone who didn't wouldn't be a good candidate to be an engineer.
So it does not follow that some courses having difficulty making their numbers means that there is a crisis of recruitment in engineering courses. This might be a crisis for those working in the Civ Eng department, but if students don't care, and employers already have twice as many candidate as they need, why should engineering care? If we end up with too few civil engineers, it will presumably become better paid, but the laws of supply and demand suggest there are too many of them at present.
As well as the differences between disciplines, there are accredited and unaccredited courses, and courses at higher and lower status institutions. If the unaccredited courses at third rate institutions intended to cheat ill-informed students of their money (because institution and course status is often more important in the UK job market than degree classification) have difficulties filling their courses, this need not trouble Imperial College.
Employers have a vested interest in oversupply of labour, and even good universities will to some extent lay on courses for anyone willing to pay. Law now produces six times as many graduates as there are jobs as solicitors and barristers, and wages (though not fees) have crashed. We trained far too many pharmacists in recent years and what was a secure and well paid job has become akin to that of a shop assistant for many of those lucky enough to have jobs at all.
So when we strip out the unhelpful generalizations, special pleading and propaganda from vested interests, we see the following:
There is no general shortage of graduate engineers (or both employment rates and wages for graduates would be higher)
There may be local shortages of certain kinds of engineers and engineering students (but paying professional engineers more money can fix them)
Training more engineers will on the other hand not fix this problem, as it will (by increasing supply in a market which is already oversupplied) decrease both wages and the chances of employment for graduates and hence the attractiveness of undertaking our challenging and expensive courses.
Encouraging women to study engineering will not fix this problem (as there is no evidence that the profession suffers in any way because women generally prefer medicine to engineering)
It seems from Chem Eng's example that if it is politically desirable to increase the number of women in engineering education, paying engineers more seems to work.
So it looks to me as if much of what is presently being done is entirely wrong-headed, based in an uncritical acceptance of political propaganda.
We can however always use more good engineers. Maybe if we produced more of them we might once more have an economy based on designing and making things, driven by these good engineers.
In my opinion, many of the half of engineering graduates who get jobs as engineers should think themselves lucky. More than 75% of them will never be good engineers. I have seen them in education, and I have seen them in practice, and they are just making up the numbers in my opinion.
I cannot tell reliably why this is. Have we already dipped to the bottom of the pool of natural engineers? Are engineers born or made? How can we reliably measure ingenuity? Can we foster it, and if so how? Are our present metrics of quality well correlated with ingenuity?
The answers to these questions are the key to making more of the good engineers we all think are needed. If we wasted less time on political agendas we'd have more time to find answers to them.