THE big problem in the philosophy of science is: Where do valid hypotheses come from?
The problem is that the 'search space' for valid hypotheses is vast, and nearly all possible hypotheses are wrong.
So how do creative scientists discover true hypotheses - that is, hypotheses that are basically valid explanations, suitable for research and development by standard methods?
Another problem is that most of what gets called science is not primarily creative, but uses already-made/ discovered hypotheses to extrapolate and interpolate (fill the gaps between) their assumptions, by deploying fairly standard methods of 'trial and error' in the domain of 'research and development'.
In other words, creative science corresponds closely with what Thomas Kuhn termed 'revolutionary' science (although the creative and paradigm-changing 'revolution' is mostly at a much smaller and more specialized scale than the example Kuhn used); while non-creative science corresponds to what Kuhn termed 'normal' science.
In practice, I think creative science uses something like the following 'method':
1. Focus
2. Effort
3. Honesty
Focus: The scientists focuses on some problem or topic - this is in one sense chosen by the scientist; but in another sense the problem chooses the scientist: he is firstly gripped by a problem and secondly consents to work on it and thirdly he is able to do this.
Effort: The scientist then puts in a sustained and considerable effort directed at the problem. This requires motivation, since it is seldom that external influences will be pushing him to pick this specific problem, and to keep working on it in the face of distraction and discouragements.
Honesty: The scientist must be honest - he must want to know the truth, must behave truthfully, and must communicate his findings and beliefs truthfully. Creative science is difficult and rarely accomplished, and unless the individual is honest he will find it much quicker and easier to fake a discovery (more or less) than actually to make one; easier to hype or otherwise distort a discovery rather than to communicate it as clearly as possible.
Also, an attitude of honesty is necessary in order that the phenomenon comes more clearly into view; to clear-away the clouds and distortions that obscure it. The ability to know is typically prevented by wishful thinking and other types of preconception - unless there is a honest desire to know the truth, then the truth will not be known.
Given the above general set-up - which as-it-were places the scientist into a position from which creative science can be done - what then is the micro-method of the creative scientist?
1. Attention
2. Intuition
3. Imagination
What happens with creative science is that the scientists is paging attention, is absorbed-by the phenomenon; then looking within himself (intuition) he apprehends the reality of the situation by his imagination.
In effect, the 'answer' appears in the scientists imagination - or, to put it more exactly, the answer is communicated to the scientist's imagination by non-perceptual means.
Negatively expressed, creative understanding does not come via the senses (vision, hearing, touch, taste, smell); it does not come via the perceptions; it does not come from memory.
How then does the answer arrive in the imagination? I imagine it as a process akin to sympathy or empathy - in which an understanding of the phenomenon induces the same phenomenon in the imagination (or rather some kind of 'model of the phenomenon).
This entails either that the scientist has some sort of built-in prior understanding which is elicited by the process of contemplating the phenomenon (rather as Plato said that 'learning' was actually a matter of making explicit what was already known) - or else that the understanding is communicated-to the imagination from some external source (which would correspond to 'inspiration').
So, the primary source of creative understanding is located in the imagination of the scientist, and by this account it is valid - the understanding is valid, albeit neither complete nor undistorted due to the constraint of any phenomenon being more complex than the finite representation of it in the imagination).
But so far, the understanding is 'subjective' in the sense of being private, inside the mind of the scientist - the scientist must then communicate this understanding into the public domain via language - and in doing this there will inevitably be selection, distortion and also the possibility of misunderstanding from those trying to understand the phenomenon from its public description.
That is about as far as I can go in describing what happens - and it leaves open (or, at least, open to dispute) some fundamental aspects of the problem - such as where the imaginative understanding comes from and what makes it valid.
It creates the apparent paradox, that real, creative science comes from outside science - but this is surely true, in the sense that this is also found in other creative realms such as literature, music and art.
I think it would be found that the above psychological description fits the actuality of valid discovery - when such things are known. In my own experience I have found that my successes at valid creative breakthroughs - such as 'the malaise theory of depression' came after a sustained period of focused attention and contemplation of a phenomenon to which I was spontaneously drawn.
And when I failed to make a breakthrough in understanding (for example, I have not been able t understand Anorexia Nervosa) this was because I was not spontaneously interested and had failed to regard the matter with sustained attention.
Plus, of course, there is no guarantee of my reaching an imaginative understanding of any given topic - since I may have may have been insufficient attention so far (and the breakthrough still lies in the future), or I may personally be incapable of understanding for numerous reasons - or the phenomenon may be too difficult for anyone to understand, or I may have failed to be sufficiently honest and devoted in the nature of my contemplation.
But my point here is that the true 'scientific method' for creative discovery is not an algorithm or protocol - following which will yield breakthroughs; but rather a psychological, and indeed somewhat mystical, thing.
And that it involves a match-up between the scientist and the phenomenon which could be termed 'destiny' - a scientist may have a destiny to solve a particular problem, or a limited set of problems - but others will not be possible for him, since he lacks not just competence, but the necessary deep-motivation absolutely required in order to embark on focused, effortful, sustained contemplation of a particular phenomenon - a thing that may be discovered, but cannot be manufactured or imposed.
