stochastic process is probabilities
a stochastic process is a process that is more likely or less likely to happen depending on the given set of parameters [elements, factors, list of ..., list of requirement, list of undefined and undetermined requirement, ...] [sub-parameters, sub-elements, sub-factors, sub-requirement] of the situation
probabilities is having the quality (personality) of likelihood,
likely [being more or less common depending on the situation],
common [have greater chance of seeing],
conditional [being more or less likely depending on the situation],
leaving room for doubt [lack of certainty],
not set in stone [in comparison with writing in the sand],
interdependency, dependency, not independent
____________________________________
- cause and effect (the likely opposite of)
- deterministic, finite state machine
- what does random mean - complex system
- unable - in ability to account for all the causes, effects [direct, indirect, and side effects], and interactive factors (elements) in the outcome
- probabilities, statistics, look-up table
- pattern, no discern able pattern
- unpredictable, un predict able
- The three fundamentally equivalent: information, randomness, and complexity [Andrei Nikolaevich Kolmogorov], p.337, James Gleick., The information : a history, a theory, a flood, 2011.
____________________________________
- what we would like to do is this
- we would like to avoid using a word that is undefined
- a word that do not have enough real world examples - prototype
- a word that we do not know
- we would like to avoid using a word or a label
- we would like to avoid giving the impression of knowing a thing
- when in fact, we know not a thing about it [this random ness]
- by assigning the idea (notion, imaginary, fantasy, fiction) a label
- “Oh, randomness is just an excuse for ignorance”, quips Lynn Margulis., (p.317, Kevin Kelly, out of control, 1994, filename: ooc-mf.pdf )
____________________________________
James Gleick., The information : a history, a theory, a flood, 2011
p.225
A stochastic process is neither deterministic (the next event can be calculated with certainty) nor random (the next event is totally free). It is governed by a set of probabilities. Each event has a probability that depends on the state of the system and perhaps also on its previous history. If for event we substitute symbol, then a natural written language like English or Chinese is a stochastic process. So it digitized speech; so is television signal.
p.226
A message, as Shannon saw, can behave like a dynamical system whose future course is conditioned by its past history.
pp.226-227
(These he drew from a book newly published for such purposes by Cambridge university press: 100,000 digits for three shillings nine pence, and the authors “have furnished a guarantee of the random arrangement.”)
• “zero-order approximation” ── that is, random characters, no structure or correlations.
• first order ── each character is independent of the rest,
• second order ── digram, or letter pair.
(Shannon found the necessary statistics in tables constructed for use by code breakers. The most common digram in English is th, with a frequency of 168 per thousand words, followed by he, an, re, and er. Quite a few digrams have zero frequency.)
• third order ── trigram structure.
• first-order word approximation.
• second-order word approximation ── now pairs of words
p.337
The three fundamentally equivalent: information, randomness, and complexity
(The information : a history, a theory, a flood / James Gleick., 1. information science--history., 2. information society., Z665.G547 2011, 020.9--dc22, 2011, )
____________________________________
stochastic
sto|chas|tic
stochastic adj.
having to do with a random variable or variable; involving chance or probability: Stochastic processes arise in physics, astronomy, economics, genetics, ecology, and many other fields of science. The simplest and most celebrated example of a stochastic process is the Brownian motion of a particle (Scientific American)
stochastically, adv.
sto|chas'ti|cal|ly
Greek stóchos aim, guess
stochastic random variable or variables, chance or probabilities, aim, guess
usage - stochastic process or stochastic processes
____________________________________
Ed Catmull with Amy Wallace, creativity, inc., 2014 [ ]
p.155
Yet randomness remains stubbornly difficult to understand.
The problem is that our brain aren't wired to think about it. Instead, we are built to look for patterns in sights, sounds, interactions, and events in the world. This mechanism is so ingrained that we see patterns even when they aren't there. There is a subtle reason for this: We can store patterns and conclusions in our heads, but we cannot store randomness itself. Randomness is a concept that defies categorization; by definition, it comes out of nowhere and can't be anticipated. While we intellectually accept that it exists, our brains can't completely grasp it, so it has less impact on our consciousness than things we can see, measure, and categorize.
(creativity, inc. : overcoming the unseen forces that stand in the way of true inspiration / Ed Catmull with Amy Wallace., 1. creativity ability in business2. corporate culture, 3. organizational effectiveness, 4. pixar (firm), © 2014 by Edwin Catmull, 658.4071 Catmull, p.155)
____________________________________
James Gleick., The information : a history, a theory, a flood, 2011
p.335-336
Andrei Nikolaevich Kolmogorov
three approaches to the definition of the concept ‘Amount of Information’
He described three approaches: the combinatorial, the probabilistic, and the algorithmic.
p.337
The three fundamentally equivalent: information, randomness, and complexity
(The information : a history, a theory, a flood / James Gleick., 1. information science--history., 2. information society., Z665.G547 2011, 020.9--dc22, 2011, )
____________________________________
Douglas Hofstadter & Emmanuel Sander, Surfaces and essences: analogy as the fuel and fire of thinking, 2013
p.345
Psychologist Eleanor Rosch has described how our categories respect the correlational structure of the world. That is, not all combinations of properties are equally likely; rather, certain properties tend to co-occur in our environment.
p.345
For example, the property of flying is coorelated with the properties of laying eggs and having a beak. In other words, when we perceive surface-level features, that activates in our minds other features that are correlated with those first features. These secondarily activated features are ones that our experience tells us tend to be present when the first ones are, but in themselves they are not instantly perceptible. In this way, what we see on something's surface leads us to its hidden depths, and thus allows us to draw meaningful, insight-lending analogies with what we have known before.
p.345
Psychologist Myriam Bassok has carefully studied this notion of “induced structure”, focusing on how it applies to the way that students learn in schools. Furthermore, the relevance of her finding goes well beyond the educational system; indeed, they apply to the way that we relate to the world around us. Thus a feather is likely to be light; a peach is likely to have a pit; something round stands a fair chance of being able to roll; and kicking something that looks like an anvil is not, in general, highly advisable.
(Surfaces and essences: analogy as the fuel and fire of thinking, Douglas Hofstadter & Emmanuel Sander, 2013, )
____________________________________
1:52:09
Neuroscience 2013 Dialogues Between Neuroscience and Society: The Creative Culture
https://youtu.be/fwajgXkaOoo?t=543
https://youtu.be/fwajgXkaOoo?t=543
start of talk
https://youtu.be/fwajgXkaOoo?t=375
https://youtu.be/fwajgXkaOoo?t=375
https://www.youtube.com/watch?v=fwajgXkaOoo
https://www.youtube.com/watch?v=fwajgXkaOoo
59:03 thank you very much
Society for Neuroscience
Published on Jan 23, 2014
Ed Catmull, president of Pixar and Walt Disney Animation Studios, spoke about creativity at Neuroscience 2013.
([ telling story is a way of communicating, ])
([ teaching, and connecting with emotion ])
([ the magician, the illusionist ])
([ we are the one that is creating the illusion ])
([ the magician is taking advantage of knowing ])
([ how we - meaning the brain - create illusion ])
([ this is what we do with films ])
https://youtu.be/fwajgXkaOoo?t=2771
https://youtu.be/fwajgXkaOoo?t=2771
([ stochastic self-similarity ])
([ stochastic self-similarities ])
([ stochastic self-similar (sss) processes ])
([ stochastic self-similar processes ])
https://youtu.be/fwajgXkaOoo?t=2428
https://youtu.be/fwajgXkaOoo?t=2428
([ but here is the trick, if any one of those ])
([ group wins, we loose ])
([ that's the balance you has to have ])
([ and you gotta understand that ])
([ and the goal in a lot of group is to win ])
([ if anyone of them win, we loose ])
https://youtu.be/fwajgXkaOoo?t=5497
https://youtu.be/fwajgXkaOoo?t=5497
____________________________________
The word “chaos” comes from the Greek word “khaos”, meaning “gaping void”. Chaos in other word means a state of utter confusion or the inherent unpredictability in the behavior of a complex natural system.
(Ancient Greek: χάος, romanized: kháos)
[Abstract]
Chaos theory is a mathematical field of study which states that non-linear dynamical systems that are seemingly random are actually deterministic from a much simpler equations.
introduced to the modern world by Edward Lorenz in 1972
____________________________________
mathematicians and physicists have overlooked dynamical systems as random and unpredictable.
linear, that is to say, systems that follow predictable patterns and arrangement.
There were too many things going on to keep track of with linear equations.
Systems considered to be chaotic are not really chaotic at all - they are just not as predictable as the cause-and-effect kind of ideas associated with linear dynamics.
In many mythologies the creation of the universe is symbolized by the gods of order conquering chaos. “While the universe, including the gods, may originate from chaos, order seems to emerge also. Order banishes chaos but never really destroys it.”[1] Despite its etymological Greek origin ‘χάος’ (zhaos), the notion of chaos appears in many different ancient narrations about the origins of the world.
What is Chaos theory?
Chaos theory is the study of complex, nonlinear, dynamic systems.
It is a branch of mathematics that deals with systems that appear to be orderly (deterministic) but, in fact, harbor chaotic behaviors. It also deals with systems that appear to be chaotic, but, in fact, have underlying order. Chaos theory studies the behavior of dynamical systems that are highly sensitive to initial conditions.
these systems are deterministic, meaning that their future behavior is fully determined by their initial conditions, with no random elements involved. In other words, the deterministic nature of these systems does not make them predictable.
Edward Lorenz
1956 paper, Deterministic Nonperiodic Flow
the principle of Sensitive Dependence on Initial Conditions (SDIC), which is now viewed as a key component in any chaotic system.
1972 talk, “Predictability: does the flap of a butterfly's wings in Brazil set off a tornado in Texas”
The basic principle is that even in an entirely deterministic system the slightest change in the initial data can cause abrupt and seemingly random changes in the outcome.
____________________________________
(Ancient Greek: χάος, romanized: kháos)
The presence of chaotic systems in nature seems to place a limit on our ability to apply deterministic physical laws to predict motions with any degree of certainty.
The discovery of chaos seems to imply that randomness lurks at the core of any deterministic model of the universe.11
____________________________________
Random Number Generation
The programming community is majorly interested in design and maintaenance of standard pseudo-random number generators that are portable and reusable. The pseudo-random number generators, familiar to all programmers, are derived from deterministic chaotic dynamical systems. As we shall see, when pseudo-random number generators are designed properly, the sequences produced are completely predictable.
Random number generation has been of great interest from the beginnings of computing. Philosophically and mathematically, the concept of randomness poses many problems. Intuitively, we equate randomness with unpredict ability.
In the past 25 years, largely due to the separate efforts of Chaitin and Kolmogorov the concept of randomness has been made definitive. They accomplished this by developing the concept of algorithmic complexity. The complexity of an n-digit sequence is the length in bits of the shortest computer program that can produce the sequence. For a very regular sequence, 1111111111, for example, a very small program is needed. As the sequence becomes highly irregular and as the length of the sequence grows beyond bounds, it can be shown that the shortest program needed to produce the sequence is slightly larger than the sequence itself.
Clearly, any algorithmic implementation of the theoretical ideal of randomness on a computer will be imperfect. From all view point above, a random sequence is a non-computable infinite sequence. A measure of the “goodness” of a pseudo-random number generator is aperiodicity [not periodic; irregular.; periodic, periodical - happening or recurring at regular intervals, the tendency to recur at regular intervals; aperiodic - the tendency to NOT occur at regular intervals].
However, any finite computer algorithm implementation yields only periodic sequences--although of very long period. Thus, random number generators found in computer languages are referred to as “pseudo-random” number generators.
[de facto standard] is a custom or convention that has achieved a dominant position by public acceptance or market forces. De facto is a Latin phrase that means in fact (literally [by or from fact]) in the sense of "in practice but not necessarily ordained by law" or "in practice or actuality, but not officially established", as opposed to de jure.
In 1951, Lehmer proposed an algorithm that has become the de facto standard pseudo-random number generator. As it is usually implemented, the algorithm is known as a Prime Modulus Multiplicative Linear Congruential Generator. It is better known as a Lehmer generator.
____________________________________
It's well known that the heart has to be largely regular or you die. But the brain has to be largely irregular; if not, you have epilepsy. This shows that irregularity, chaos, leads to complex systems. It's not all disorder. On the contrary, I would say chaos is what makes life and intelligence possible.
- Ilya Prigogine
Chaotic systems are very sensitive to the initial conditions which means that a slight change in the starting point can lead to enormously different outcomes [over the longer-term]. This makes the system fairly unpredictable [mathematically]. Chaos systems never repeat but they always have some order. Most of the systems we find in the world predicted by classical physics are the exceptions but in this world of order, chaos rules. Chaos theory is a new way of thinking about what we have. It gives us a new concept of measurements and scales.
(Ancient Greek: χάος, romanized: kháos)
Because of chaos, it is realized that even simple systems may give rise to and, hence, be used as models for complex behavior.
Chaos forms a bridge between different fields. Chaos offers a fresh way to proceed with observational data, especially those data which may be ignored because they proved too erratic.
source:
https://www.slideshare.net/anthaceorote/chaos-theory-an-introduction
1. Gajanan Shewale
2. Nayana Shinde
3. Aditya Shirode
4. Suntej Singh
5. Jayesh Solanki
6. Madhuri Tajane
7. Gaurav Tripathi
Sardar Patel Institute of Technology, Mumbai
Communication and presentation techniques
academic year 2012-2013
____________________________________
[Applications]
Although chaos theory was born from observing weather patterns, it has become applicable to a variety of other situatiions.
[Cryptography]
Chaos theory has been used for many years in cryptography. In the past few decades, chaos and nonlinear dynamics have been used in the design of hundreds of cryptographic primitives. These algorithms include image encryption algorithms, hash functions, secure pseudo-random number generators, stream ciphers, watermarking and steganography.[103] The majority of these algorithms are based on uni-modal chaotic maps and a big portion of these algorithms use the control parameters and the initial condition of the chaotic maps as their keys.[104] From a wider perspective, without loss of generality, the similarities between the chaotic maps and the cryptographic systems is the main motivation for the design of chaos based cryptographic algorithms.[103] One type of encryption, secret key or symmetric key, relies on diffusion and confusion, which is modeled well by chaos theory.[105] Another type of computing, DNA computing, when paired with chaos theory, offers a way to encrypt images and other information.[106] Many of the DNA-Chaos cryptographic algorithms are proven to be either not secure, or the technique applied is suggested to be not efficient.[107][108][109]
source:
https://en.wikipedia.org/wiki/Chaos_theory
____________________________________
http://www.empiricalzeal.com/2012/12/21/what-does-randomness-look-like/
DECEMBER 21, 2012 · 4:48 PM ↓ Jump to Comments
What does randomness look like?
800px-V-1_cutaway
On 13 June 1944, a week after the allied invasion of Normandy, a loud buzzing sound rattled through the skies of battle-worn London. The source of the sound was a newly developed German instrument of war, the V-1 flying bomb. A precursor to the cruise missile, the V-1 was a self-propelled flying bomb, guided using gyroscopes, and powered by a simple pulse jet engine that gulped air and ignited fuel 50 times a second. This high frequency pulsing gave the bomb its characteristic sound, earning them the nickname buzzbombs.
From June to October 1944, the Germans launched 9,521 buzzbombs from the coasts of France and the Netherlands, of which 2,419 reached their targets in London. The British worried about the accuracy of these aerial drones. Were they falling haphazardly over the city, or were they hitting their intended targets? Had the Germans really worked out how to make an accurately targeting self-guided bomb?
Fortunately, they were scrupulous in maintaining a bomb census, that tracked the place and time of nearly every bomb that was dropped on London during World War II. With this data, they could statistically ask whether the bombs were falling randomly over London, or whether they were targeted. This was a math question with very real consequences.
Imagine, for a moment, that you are working for the British intelligence, and you’re tasked with solving this problem. Someone hands you a piece of paper with a cloud of points on it, and your job is to figure out if the pattern is random.
Let’s make this more concrete. Here are two patterns, from Steven Pinker’s book, The Better Angels of our Nature. One of the patterns is randomly generated. The other imitates a pattern from nature. Can you tell which is which?
pinker-glow-worms-and-stars-plot
Thought about it?
Here is Pinker’s explanation.
The one on the left, with the clumps, strands, voids, and filaments (and perhaps, depending on your obsessions, animals, nudes, or Virgin Marys) is the array that was plotted at random, like stars. The one on the right, which seems to be haphazard, is the array whose positions were nudged apart, like glowworms
That’s right, glowworms. The points on the right records the positions of glowworms on the ceiling of the Waitomo cave in New Zealand. These glowworms aren’t sitting around at random, they’re competing for food, and nudging themselves away from each other. They have a vested interest against clumping together.
Update: Try this out for yourself. After reading this article, praptak and roryokane over at hacker news wrote a script that will generate random and uniform distributions in your browser, nicely illustrating the point.
Try to uniformly sprinkle sand on a surface, and it might look like the pattern on the right. You’re instinctively avoiding places where you’ve already dropped sand. Random processes have no such prejudices, the grains of sand simply fall where they may, clumps and all. It’s more like sprinkling sand with your eyes closed. They key difference is that randomness is not the same thing as uniformity. True randomness can have clusters, like the constellations that we draw into the night sky.
Here’s another example. Imagine a professor asks her students to flip a coin 100 times. One student diligently did the work, and wrote down their results. The other student is a bit of a slacker, and decided to make up fake coin tosses instead of doing the experiment. Can you identify which student is the slacker?
Student 1:
THHHTHTTTTHTTHTTTHHTHTTHT
HHHTHTHHTHTTHHTTTTHTTTHTH
TTHHTTTTTTTTHTHHHHHTHTHTH
THTHTHHHHHTHHTTTTTHTTHHTH
Student 2:
HTTHTTHTHHTTHTHTHTTHHTHTT
HTTHHHTTHTTHTHTHTHHTTHTTH
THTHTHTHHHTTHTHTHTHHTHTTT
HTHHTHTHTHTHHTTHTHTHTTHHT
Take a moment to reason this through.
The first student’s data has clusters – long runs of up to eight tails in a row. This might look surprising, but it’s actually what you’d expect from random coin tosses (I should know – I did a hundred coin tosses to get that data!) The second student’s data in suspiciously lacking in clusters. In fact, in a hundred coin tosses, they didn’t get a single run of four or more heads or tails in a row. This has about a 0.1% chance of ever happening, suggesting that the student fudged the data (and indeed I did).
Image by Scott Adams.
Trying to work out whether a pattern of numbers is random may seem like an arcane mathematical game, but this couldn’t be further from the truth. The study of random fluctuations has its roots in nineteenth century French criminal statistics. As France was rapidly urbanizing, population densities in cities began to shoot up, and crime and poverty became pressing social problems.
371px-Adolphe_Quételet_by_Joseph-Arnold_Demannez
In 1825, France began to collect statistics on criminal trials. What followed was perhaps the first instance of statistical analysis used to study a social problem. Adolphe Quetelet was a Belgian mathematician, and one of the early pioneers of the social sciences. His controversial goal was to apply probability ideas used in astronomy to understand the laws that govern human beings.
In the words of Michael Maltz,
In finding the same regularity in crime statistics that was found in astronomical observations, he argued that, just as there was a true location of a star (despite the variance in the location measurements), there was a true level of criminality: he posited the construct of l’homme moyen (the “average man”) and, moreover, l’homme moyen moral. Quetelet asserted that the average man had a statistically constant “penchant for crime,” one that would permit the “social physicist” to calculate a trajectory over time that “would reveal simple laws of motion and permit prediction of the future” (Gigerenzer et al, 1989).
Quetelet noticed that the conviction rate of criminals was slowly falling over time, and deduced that there must be a downward trend in the “penchant for crime” in French citizens. There were some problems with the data he used, but the essential flaw in his method was uncovered by the brilliant French polymath and scientist Siméon-Denis Poisson.
Simeon_Poisson
Poisson’s idea was both ingenious and remarkably modern. In today’s language, he argued that Quetelet was missing a model of his data. He didn’t account for how jurors actually came to their decisions. According to Poisson, jurors were fallible. The data that we observe is the rate of convictions, but what we want to know is the probability that a defendant is guilty. These two quantities aren’t the same, but they can be related. The upshot is that when you take this process into account, there is a certain amount of variation inherent in conviction rates, and this is what one sees in the French crime data.
In 1837, Poisson published this result in “Research on the Probability of Judgments in Criminal and Civil Matters“. In that work, he introduced a formula that we now call the Poisson distribution. It tells you the odds that a large number of infrequent events result in a specific outcome (such as the majority of French jurors coming to the wrong decision). For example, let’s say that on average, 45 people are struck by lightning in a year. Feed this in to Poisson’s formula, along with the population size, and it will spit out the odds that, say, 10 people will be struck by lightning in a year, or 50, or a 100. The assumption is that lightning strikes are independent, rare events that are just as likely to occur at any time. In other words, Poisson’s formula can tell you the odds of seeing unusual events, simply due to chance.
By Randall Munroe
One of the first applications of Mr. Poisson’s formula came from an unlikely place. Leap sixty years ahead, over the Franco-Prussian war, and land in 1898 Prussia. Ladislaus Bortkiewicz, a Russian statistician of Polish descent, was trying to understand why, in some years, an unusually large number of soldiers in the Prussian army were dying due to horse-kicks. In a single army corp, there were sometimes 4 such deaths in a single year. Was this just coincidence?
A single incidence of death by horse kick is rare (and assumedly independent, unless the horses have a hidden agenda). Bortkiewicz realized that he could use Poisson’s formula to work out how many deaths you expect to see. Here is the prediction, next to the real data.
Number of Deaths by Horse Kick in a year Predicted Instances (Poisson) Observed Instances
0 108.67 109
1 66.29 65
2 20.22 22
3 4.11 3
4 0.63 1
5 0.08 0
6 0.01 0
See how well they line up? The sporadic clusters of horse-related deaths are just what you would expect if horse-kicking was a purely random process. Randomness comes with clusters.
By Ryan North
I decided to try this out for myself. I looked for publicly available datasets for deaths due to rare events, and came across the International Shark Attack File, that tabulates worldwide incidents of sharks attacking people. Here’s the data of shark attacks in South Africa.
Year Number of Shark Attacks in South Africa
2000 4
2001 3
2002 3
2003 2
2004 5
2005 4
2006 4
2007 2
2008 0
2009 6
2010 7
2011 5
The numbers are fairly low, with an average of 3.75. But compare 2008 and 2009. One year has zero shark attacks, and the next has 6. And then in 2010, there are 7. You can already imagine the headlines crying out, “Attack of the sharks!“. But is there really a shark rebellion, or would you expect to see these clusters of shark attacks due to chance? To find out, I compared the data to Mr. Poisson’s prediction.
shark_attacks_south_africa
“Anyone else see the shark fin?” Nice catch, by @Gareth_Elms
In blue are the observed counts of years with a 0,1,2,3.. shark attacks. For example, the long blue bar represents the 3 years in which there were 4 shark attacks (2000, 2005 and 2006). The red dotted line is the Poisson distribution, and it represents the outcomes that you would expect if the shark attacks were a purely random process. It fits the data well – I found no evidence of clustering beyond what is expected by a Poisson process (p=0.87). I’m afraid this rules out the great South African shark uprising of 2010. The lesson, again, is that randomness isn’t uniform.
Which brings us back to the buzzbombs. Here’s a visualization of the number of bombs dropped over different parts, reconstructed by Charles Franklin using the original maps in the British Archives in Kew.
london buzzbomb distribution
Note: A clarification. The plot above shows the distribution of bombs that were dropped over London. The question I’m asking is, if you zoom in to the part of the city most heavily under attack (essentially the mountain that you see in the figure above), are the bombs being guided more precisely, to hit specific targets?
It’s far from a uniform distribution, but does it show evidence of precise targeting? At this point, you can probably guess how to answer this question. In a report titled An Application of the Poisson Distribution, a British statistician named R. D. Clarke wrote,
During the flying-bomb attack on London, frequent assertions were made that the points of impact of the bombs tended to be grouped in clusters. It was accordingly decided to apply a statistical test to discover whether any support could be found for this allegation.
Clarke took a 12 km x 12 km heavily bombed region of South London, and sliced it up in to a grid. In all, he divided it into 576 squares, each about the size of 25 city blocks. Next, he counted the number of squares with 0 bombs dropped, 1 bomb dropped, 2 bombs dropped, and so on.
In all, 537 bombs fell over these 576 squares. That’s a little under one bomb falling per square, on average. He plugged this number into Poisson’s formula, to work out how much clustering you would expect to see by chance. Here’s the relevant table from his paper:
poisson table buzzbombs
Compare the two columns, and you can see how incredibly close the prediction comes to reality. There are 7 squares that were hit by 4 bombs each – but this is what you would expect by chance. Within a large area of London, the bombs weren’t being targeted. They rained down at random in a devastating, city-wide game of Russian roulette.
The Poisson distribution has a habit of creeping up in all sorts of places, some inconsequential, and others life-altering. The number of mutations in your DNA as your cells age. The number of cars ahead of you at a traffic light, or patients in line before you at the emergency room. The number of typos in each of my blog posts. The number of patients with leukemia in a given town. The numbers of births and deaths, marriages and divorces, or suicides and homicides in a given year. The number of fleas on your dog.
From mundane moments to matters of life and death, these Victorian scientists have taught us that randomness plays a larger role in our lives than we care to admit. Sadly, this fact offers little consolation when the cards in life fall against your favor.
“So much of life, it seems to me, is determined by pure randomness.” – Sidney Poitier
References
Shark attacks and the Poisson approximation. A nice introduction to using Poisson’s formula, with applications including the birthday paradox, one of my favorite examples of how randomness is counter-intuitive.
From Poisson to the Present: Applying Operations Research to Problems of Crime and Justice. A good read about the birth of operations research as applied to crime.
Applications of the Poisson probability distribution. Includes a list of many applications of the Poisson distribution.
Steven Pinker’s book The Better Angels of our Nature has many great examples of how our intuition about randomness is generally wrong.
Want to know more about the accuracy of the flying bombs? The story is surprisingly rich, involving counterintelligence and espionage. Here’s a teaser.
____________________________________
Flight over Wall St
19 April 1997
By John Casti
Santa Fe, New Mexico
ON 29 March 1900, in a dusty seminar room at the Sorbonne in Paris, the renowned mathematician Jules Henri Poincaré was presiding as a student defended a slightly unusual doctoral dissertation. In Poincaré`s words, the topic was “somewhat remote from those our candidates are in the habit of treating”. This topic was a mathematical treatment of how prices for French government bonds and their options fluctuated on the Paris Bourse. The author of this dissertation, Louis Bachelier, received the insultingly low grade of mention honorable rather than the more usual mention très honorable, the level needed to be taken seriously as a candidate for an academic post in France. Perhaps the dissertation’s title, The Theory of Speculation, as well as its frankly commercial character had something to do with the disdain with which the examiners viewed Bachelier’s efforts. Who can say? What we do know is that this was pioneering work, the academic community’s initial salvo in the battle to unlock the secrets of speculative markets. And Bachelier’s ideas were truly visionary, for a century later his conclusions remain at the heart of the dominant paradigm as to how prices fluctuate in such markets.
[changes in prices]
Bachelier focused his mathematical artillery on changes in prices, rather than on the prices themselves. Prices sometimes go up, of course, and sometimes they go down. But by studying the ups and downs with care, Bachelier discovered that he could say more than that.
[The ups and downs]
First, he noticed that price changes over one interval have nothing to do with changes over another. If a stock price goes …
Read more: https://www.newscientist.com/article/mg15420784-700-flight-over-wall-st/
source:
https://www.newscientist.com/article/mg15420784-700-flight-over-wall-st/
https://www.newscientist.com/article/mg15420784.700-flight-over-wall-st.html/
____________________________________
Mario Livio, Brilliant blunders, 2013 [ ]
p.210
Penzias and Wilson were working at the Bell Telephone Laboratories in New Jersey with an antenna built for communication satellites. To their annoyance, they were picking up some sort of pervasive background radio noise: microwave radiation that appeared to be the same from all directions. After failing to explain away this disturbing “hiss” as an instrumental artifact, Penzias and Wilson finally announced the detection of an intergalatic temperature excess of about 3 Kelvin (3 degrees above absolute zero).
p.210
Lacking the necessary background, Penzias and Wilson did not realize initially what they had found. Robert Dicke of Princeton University, however, recognized the signal immediately. Dicke was in the process of building a radiometer to search for the relic radiation from the big bang, previously predicted by Alpher, Hermann, and Gamow.
p.210
Consequently, his correct interpretation of the results of Penzias and Wilson literally transformed the big bang theory from hypothesis into experimentally tested physics.
p.210
As the universe expanded, the incredibly hot, dense, and opaque fireball coolded down continuously, eventually reaching its present temperature of about 2.7 Kelvin.
p.210
Since then, observations of the cosmic microwave background have produced some of the most precise measurement in cosmology.
p.210
its intensity changes with wavelength precisely as expected from a thermal source
(Brilliant blunders: from Darwin to Einstein ─ colossal mistakes by great scientists that changed our understanding of life and the universe / Mario Livio., 1. errors, scientific., Q172.5.E77L58 2013, 500─dc23, first Simon & Schuster hardcover edition May 2013, 2013, )
____________________________________
Kevin Kelly, out of control, 1994
p.315
“Natural selection is the editor, not the author,” says Lynn Margulis.
p.316
the Krebs cycle is the basic fuel plant in every cell in your body. It has worked fine for hundreds of millions of years. There is simply too little to gain, and far too much to lose, in fiddling with it now. When a variation is detected in the code for the Krebs cycle, it is quickly extinguished. On the other hand, body size and body proportions might be worth tweaking; let's leave that area open to variation.
p.317
“Oh, randomness is just an excuse for ignorance”, quips Lynn Margulis.
(Kevin Kelly, out of control, 1994, filename: ooc-mf.pdf )
____________________________________
Beer, 1959 (Stafford Beer)
*** [ Stafford Beer's classification of systems, p.171, George P. Richardson, 1991, Feedback thought in social science and systems theory ]
Stafford Beer's classification of systems
[p.171]
-----------------------------------------------------------------------
SYSTEMS Simple Complex Exceedingly
complex
-----------------------------------------------------------------------
Deterministic Window catch Electronic digital EMPTY
computer
--------------------------------------
Billiards Planetary system
--------------------------------------
Machine-shop Automation
lay-out
-----------------------------------------------------------------------
Probabilistic Penny tossing Stockholding The economy
---------------------------------------------------------
Jellyfish Conditioned The brain
movement reflexed
---------------------------------------------------------
Statistical Industrial THE COMPANY
quality control profitability
FIGURE 4.1: Stafford Beer's classification of systems based on degrees of complexity and uncertainty. Source: Beer (1959, p. 18).
Beer, Stafford (1959/1967). Cybernetics and Management (London: English Universities Press).
(Richardson, George P., Feedback thought in social science and systems theory, copyright © 1991 by the University of Pennsylvania Press)
(Feedback thought in social science and systems theory / George P. Richardson (1991), 1. social science--methodology., 2. feedback control systems., p.171 )
____________________________________
BANK OF ENGLAND
Speech
Tails of the unexpected
Paper by Andrew G Haldane (and) Benjamin Nelson
Given at “The Credit Crisis Five Years On: Unpacking the Crisis”, conference held at the University of Edinburg Business School, 8-9 June
8 June 2012
•••• ••• ••••
But as Nassim Taleb reminded us, it is possible to be Fooled by Randomness (Taleb, 2001). For Taleb, the origin of this mistake was the ubiquity in economics and finance of a particular way of describing the distribution of possible real world outcomes. For non-nerds, this distribution is often called the bell-curve. For nerds, it is the normal distribution. For nerds who like to show-off, the distribution is Gaussian.
[normal distribution, bell-curve, Gaussian distribution]
The normal distribution provides a beguilingly simple description of the world. Outcomes lie symmetrically around the mean, with a probability that steadily decays. It is well-known that repeated games of chance deliver random outcomes in line with this disbribution: tosses of a fair coin, sampling of coloured balls from a jam-jar, bets on a lottery number, games of paper/scissors/stone. Or have you been fooled by randomness?
•••• ••• ••••
All speeches are available online at
http://www.bankofengland.co.uk/publications/Pages/speeches/default.aspx
<put additional URL here>
https://drive.google.com/file/d/1A18Z9AkjaUqQFs4HX_j32NqhU7IlrKPU/view?usp=sharing
https://drive.google.com/file/d/1A18Z9AkjaUqQFs4HX_j32NqhU7IlrKPU/
____________________________________
- probabilities, statistics, distribution curve
- probabilities table, statistics table, look-up table
- distribution table, frequency table, data table
- [normal distribution, bell-curve, Gaussian distribution]
- In statistics, curves like that are called “long-tailed distribution”, because the tail of the curve is very long relative to the head., (p.10, Chris Anderson, The long tail, 2006)
____________________________________
Technology
Computer Sciences
October 16, 2015
System that replaces human intuition with algorithms outperforms human teams
by Larry Hardesty, Massachusetts Institute of Technology
http://phys.org/news/2015-10-human-intuition-algorithms-outperforms-teams.html
http://groups.csail.mit.edu/EVO-DesignOpt/groupWebSite/uploads/Site/DSAA_DSM_2015.pdf
Big-data analysis consists of searching for buried patterns that have some kind of predictive power. But choosing which "features" of the data to analyze usually requires some human intuition.
"What we observed from our experience solving a number of data science problems for industry is that one of the very critical steps is called feature engineering," Veeramachaneni says. "The first thing you have to do is identify what variables to extract from the database or compose, and for that, you have to come up with a lot of ideas."
____________________________________
____________________________________
a stochastic process is a process that is more likely or less likely to happen depending on the given set of parameters [elements, factors, list of ..., list of requirement, list of undefined and undetermined requirement, ...] [sub-parameters, sub-elements, sub-factors, sub-requirement] of the situation
probabilities is having the quality (personality) of likelihood,
likely [being more or less common depending on the situation],
common [have greater chance of seeing],
conditional [being more or less likely depending on the situation],
leaving room for doubt [lack of certainty],
not set in stone [in comparison with writing in the sand],
interdependency, dependency, not independent
____________________________________
- cause and effect (the likely opposite of)
- deterministic, finite state machine
- what does random mean - complex system
- unable - in ability to account for all the causes, effects [direct, indirect, and side effects], and interactive factors (elements) in the outcome
- probabilities, statistics, look-up table
- pattern, no discern able pattern
- unpredictable, un predict able
- The three fundamentally equivalent: information, randomness, and complexity [Andrei Nikolaevich Kolmogorov], p.337, James Gleick., The information : a history, a theory, a flood, 2011.
____________________________________
- what we would like to do is this
- we would like to avoid using a word that is undefined
- a word that do not have enough real world examples - prototype
- a word that we do not know
- we would like to avoid using a word or a label
- we would like to avoid giving the impression of knowing a thing
- when in fact, we know not a thing about it [this random ness]
- by assigning the idea (notion, imaginary, fantasy, fiction) a label
- “Oh, randomness is just an excuse for ignorance”, quips Lynn Margulis., (p.317, Kevin Kelly, out of control, 1994, filename: ooc-mf.pdf )
____________________________________
James Gleick., The information : a history, a theory, a flood, 2011
p.225
A stochastic process is neither deterministic (the next event can be calculated with certainty) nor random (the next event is totally free). It is governed by a set of probabilities. Each event has a probability that depends on the state of the system and perhaps also on its previous history. If for event we substitute symbol, then a natural written language like English or Chinese is a stochastic process. So it digitized speech; so is television signal.
p.226
A message, as Shannon saw, can behave like a dynamical system whose future course is conditioned by its past history.
pp.226-227
(These he drew from a book newly published for such purposes by Cambridge university press: 100,000 digits for three shillings nine pence, and the authors “have furnished a guarantee of the random arrangement.”)
• “zero-order approximation” ── that is, random characters, no structure or correlations.
• first order ── each character is independent of the rest,
• second order ── digram, or letter pair.
(Shannon found the necessary statistics in tables constructed for use by code breakers. The most common digram in English is th, with a frequency of 168 per thousand words, followed by he, an, re, and er. Quite a few digrams have zero frequency.)
• third order ── trigram structure.
• first-order word approximation.
• second-order word approximation ── now pairs of words
p.337
The three fundamentally equivalent: information, randomness, and complexity
(The information : a history, a theory, a flood / James Gleick., 1. information science--history., 2. information society., Z665.G547 2011, 020.9--dc22, 2011, )
____________________________________
stochastic
sto|chas|tic
stochastic adj.
having to do with a random variable or variable; involving chance or probability: Stochastic processes arise in physics, astronomy, economics, genetics, ecology, and many other fields of science. The simplest and most celebrated example of a stochastic process is the Brownian motion of a particle (Scientific American)
stochastically, adv.
sto|chas'ti|cal|ly
Greek stóchos aim, guess
stochastic random variable or variables, chance or probabilities, aim, guess
usage - stochastic process or stochastic processes
____________________________________
Ed Catmull with Amy Wallace, creativity, inc., 2014 [ ]
p.155
Yet randomness remains stubbornly difficult to understand.
The problem is that our brain aren't wired to think about it. Instead, we are built to look for patterns in sights, sounds, interactions, and events in the world. This mechanism is so ingrained that we see patterns even when they aren't there. There is a subtle reason for this: We can store patterns and conclusions in our heads, but we cannot store randomness itself. Randomness is a concept that defies categorization; by definition, it comes out of nowhere and can't be anticipated. While we intellectually accept that it exists, our brains can't completely grasp it, so it has less impact on our consciousness than things we can see, measure, and categorize.
(creativity, inc. : overcoming the unseen forces that stand in the way of true inspiration / Ed Catmull with Amy Wallace., 1. creativity ability in business2. corporate culture, 3. organizational effectiveness, 4. pixar (firm), © 2014 by Edwin Catmull, 658.4071 Catmull, p.155)
____________________________________
James Gleick., The information : a history, a theory, a flood, 2011
p.335-336
Andrei Nikolaevich Kolmogorov
three approaches to the definition of the concept ‘Amount of Information’
He described three approaches: the combinatorial, the probabilistic, and the algorithmic.
p.337
The three fundamentally equivalent: information, randomness, and complexity
(The information : a history, a theory, a flood / James Gleick., 1. information science--history., 2. information society., Z665.G547 2011, 020.9--dc22, 2011, )
____________________________________
Douglas Hofstadter & Emmanuel Sander, Surfaces and essences: analogy as the fuel and fire of thinking, 2013
p.345
Psychologist Eleanor Rosch has described how our categories respect the correlational structure of the world. That is, not all combinations of properties are equally likely; rather, certain properties tend to co-occur in our environment.
p.345
For example, the property of flying is coorelated with the properties of laying eggs and having a beak. In other words, when we perceive surface-level features, that activates in our minds other features that are correlated with those first features. These secondarily activated features are ones that our experience tells us tend to be present when the first ones are, but in themselves they are not instantly perceptible. In this way, what we see on something's surface leads us to its hidden depths, and thus allows us to draw meaningful, insight-lending analogies with what we have known before.
p.345
Psychologist Myriam Bassok has carefully studied this notion of “induced structure”, focusing on how it applies to the way that students learn in schools. Furthermore, the relevance of her finding goes well beyond the educational system; indeed, they apply to the way that we relate to the world around us. Thus a feather is likely to be light; a peach is likely to have a pit; something round stands a fair chance of being able to roll; and kicking something that looks like an anvil is not, in general, highly advisable.
(Surfaces and essences: analogy as the fuel and fire of thinking, Douglas Hofstadter & Emmanuel Sander, 2013, )
____________________________________
1:52:09
Neuroscience 2013 Dialogues Between Neuroscience and Society: The Creative Culture
https://youtu.be/fwajgXkaOoo?t=543
https://youtu.be/fwajgXkaOoo?t=543
start of talk
https://youtu.be/fwajgXkaOoo?t=375
https://youtu.be/fwajgXkaOoo?t=375
https://www.youtube.com/watch?v=fwajgXkaOoo
https://www.youtube.com/watch?v=fwajgXkaOoo
59:03 thank you very much
Society for Neuroscience
Published on Jan 23, 2014
Ed Catmull, president of Pixar and Walt Disney Animation Studios, spoke about creativity at Neuroscience 2013.
([ telling story is a way of communicating, ])
([ teaching, and connecting with emotion ])
([ the magician, the illusionist ])
([ we are the one that is creating the illusion ])
([ the magician is taking advantage of knowing ])
([ how we - meaning the brain - create illusion ])
([ this is what we do with films ])
https://youtu.be/fwajgXkaOoo?t=2771
https://youtu.be/fwajgXkaOoo?t=2771
([ stochastic self-similarity ])
([ stochastic self-similarities ])
([ stochastic self-similar (sss) processes ])
([ stochastic self-similar processes ])
https://youtu.be/fwajgXkaOoo?t=2428
https://youtu.be/fwajgXkaOoo?t=2428
([ but here is the trick, if any one of those ])
([ group wins, we loose ])
([ that's the balance you has to have ])
([ and you gotta understand that ])
([ and the goal in a lot of group is to win ])
([ if anyone of them win, we loose ])
https://youtu.be/fwajgXkaOoo?t=5497
https://youtu.be/fwajgXkaOoo?t=5497
____________________________________
The word “chaos” comes from the Greek word “khaos”, meaning “gaping void”. Chaos in other word means a state of utter confusion or the inherent unpredictability in the behavior of a complex natural system.
(Ancient Greek: χάος, romanized: kháos)
[Abstract]
Chaos theory is a mathematical field of study which states that non-linear dynamical systems that are seemingly random are actually deterministic from a much simpler equations.
introduced to the modern world by Edward Lorenz in 1972
____________________________________
mathematicians and physicists have overlooked dynamical systems as random and unpredictable.
linear, that is to say, systems that follow predictable patterns and arrangement.
There were too many things going on to keep track of with linear equations.
Systems considered to be chaotic are not really chaotic at all - they are just not as predictable as the cause-and-effect kind of ideas associated with linear dynamics.
In many mythologies the creation of the universe is symbolized by the gods of order conquering chaos. “While the universe, including the gods, may originate from chaos, order seems to emerge also. Order banishes chaos but never really destroys it.”[1] Despite its etymological Greek origin ‘χάος’ (zhaos), the notion of chaos appears in many different ancient narrations about the origins of the world.
What is Chaos theory?
Chaos theory is the study of complex, nonlinear, dynamic systems.
It is a branch of mathematics that deals with systems that appear to be orderly (deterministic) but, in fact, harbor chaotic behaviors. It also deals with systems that appear to be chaotic, but, in fact, have underlying order. Chaos theory studies the behavior of dynamical systems that are highly sensitive to initial conditions.
these systems are deterministic, meaning that their future behavior is fully determined by their initial conditions, with no random elements involved. In other words, the deterministic nature of these systems does not make them predictable.
Edward Lorenz
1956 paper, Deterministic Nonperiodic Flow
the principle of Sensitive Dependence on Initial Conditions (SDIC), which is now viewed as a key component in any chaotic system.
1972 talk, “Predictability: does the flap of a butterfly's wings in Brazil set off a tornado in Texas”
The basic principle is that even in an entirely deterministic system the slightest change in the initial data can cause abrupt and seemingly random changes in the outcome.
____________________________________
(Ancient Greek: χάος, romanized: kháos)
The presence of chaotic systems in nature seems to place a limit on our ability to apply deterministic physical laws to predict motions with any degree of certainty.
The discovery of chaos seems to imply that randomness lurks at the core of any deterministic model of the universe.11
____________________________________
Random Number Generation
The programming community is majorly interested in design and maintaenance of standard pseudo-random number generators that are portable and reusable. The pseudo-random number generators, familiar to all programmers, are derived from deterministic chaotic dynamical systems. As we shall see, when pseudo-random number generators are designed properly, the sequences produced are completely predictable.
Random number generation has been of great interest from the beginnings of computing. Philosophically and mathematically, the concept of randomness poses many problems. Intuitively, we equate randomness with unpredict ability.
In the past 25 years, largely due to the separate efforts of Chaitin and Kolmogorov the concept of randomness has been made definitive. They accomplished this by developing the concept of algorithmic complexity. The complexity of an n-digit sequence is the length in bits of the shortest computer program that can produce the sequence. For a very regular sequence, 1111111111, for example, a very small program is needed. As the sequence becomes highly irregular and as the length of the sequence grows beyond bounds, it can be shown that the shortest program needed to produce the sequence is slightly larger than the sequence itself.
Clearly, any algorithmic implementation of the theoretical ideal of randomness on a computer will be imperfect. From all view point above, a random sequence is a non-computable infinite sequence. A measure of the “goodness” of a pseudo-random number generator is aperiodicity [not periodic; irregular.; periodic, periodical - happening or recurring at regular intervals, the tendency to recur at regular intervals; aperiodic - the tendency to NOT occur at regular intervals].
However, any finite computer algorithm implementation yields only periodic sequences--although of very long period. Thus, random number generators found in computer languages are referred to as “pseudo-random” number generators.
[de facto standard] is a custom or convention that has achieved a dominant position by public acceptance or market forces. De facto is a Latin phrase that means in fact (literally [by or from fact]) in the sense of "in practice but not necessarily ordained by law" or "in practice or actuality, but not officially established", as opposed to de jure.
In 1951, Lehmer proposed an algorithm that has become the de facto standard pseudo-random number generator. As it is usually implemented, the algorithm is known as a Prime Modulus Multiplicative Linear Congruential Generator. It is better known as a Lehmer generator.
____________________________________
It's well known that the heart has to be largely regular or you die. But the brain has to be largely irregular; if not, you have epilepsy. This shows that irregularity, chaos, leads to complex systems. It's not all disorder. On the contrary, I would say chaos is what makes life and intelligence possible.
- Ilya Prigogine
Chaotic systems are very sensitive to the initial conditions which means that a slight change in the starting point can lead to enormously different outcomes [over the longer-term]. This makes the system fairly unpredictable [mathematically]. Chaos systems never repeat but they always have some order. Most of the systems we find in the world predicted by classical physics are the exceptions but in this world of order, chaos rules. Chaos theory is a new way of thinking about what we have. It gives us a new concept of measurements and scales.
(Ancient Greek: χάος, romanized: kháos)
Because of chaos, it is realized that even simple systems may give rise to and, hence, be used as models for complex behavior.
Chaos forms a bridge between different fields. Chaos offers a fresh way to proceed with observational data, especially those data which may be ignored because they proved too erratic.
source:
https://www.slideshare.net/anthaceorote/chaos-theory-an-introduction
1. Gajanan Shewale
2. Nayana Shinde
3. Aditya Shirode
4. Suntej Singh
5. Jayesh Solanki
6. Madhuri Tajane
7. Gaurav Tripathi
Sardar Patel Institute of Technology, Mumbai
Communication and presentation techniques
academic year 2012-2013
____________________________________
[Applications]
Although chaos theory was born from observing weather patterns, it has become applicable to a variety of other situatiions.
[Cryptography]
Chaos theory has been used for many years in cryptography. In the past few decades, chaos and nonlinear dynamics have been used in the design of hundreds of cryptographic primitives. These algorithms include image encryption algorithms, hash functions, secure pseudo-random number generators, stream ciphers, watermarking and steganography.[103] The majority of these algorithms are based on uni-modal chaotic maps and a big portion of these algorithms use the control parameters and the initial condition of the chaotic maps as their keys.[104] From a wider perspective, without loss of generality, the similarities between the chaotic maps and the cryptographic systems is the main motivation for the design of chaos based cryptographic algorithms.[103] One type of encryption, secret key or symmetric key, relies on diffusion and confusion, which is modeled well by chaos theory.[105] Another type of computing, DNA computing, when paired with chaos theory, offers a way to encrypt images and other information.[106] Many of the DNA-Chaos cryptographic algorithms are proven to be either not secure, or the technique applied is suggested to be not efficient.[107][108][109]
source:
https://en.wikipedia.org/wiki/Chaos_theory
____________________________________
http://www.empiricalzeal.com/2012/12/21/what-does-randomness-look-like/
DECEMBER 21, 2012 · 4:48 PM ↓ Jump to Comments
What does randomness look like?
800px-V-1_cutaway
On 13 June 1944, a week after the allied invasion of Normandy, a loud buzzing sound rattled through the skies of battle-worn London. The source of the sound was a newly developed German instrument of war, the V-1 flying bomb. A precursor to the cruise missile, the V-1 was a self-propelled flying bomb, guided using gyroscopes, and powered by a simple pulse jet engine that gulped air and ignited fuel 50 times a second. This high frequency pulsing gave the bomb its characteristic sound, earning them the nickname buzzbombs.
From June to October 1944, the Germans launched 9,521 buzzbombs from the coasts of France and the Netherlands, of which 2,419 reached their targets in London. The British worried about the accuracy of these aerial drones. Were they falling haphazardly over the city, or were they hitting their intended targets? Had the Germans really worked out how to make an accurately targeting self-guided bomb?
Fortunately, they were scrupulous in maintaining a bomb census, that tracked the place and time of nearly every bomb that was dropped on London during World War II. With this data, they could statistically ask whether the bombs were falling randomly over London, or whether they were targeted. This was a math question with very real consequences.
Imagine, for a moment, that you are working for the British intelligence, and you’re tasked with solving this problem. Someone hands you a piece of paper with a cloud of points on it, and your job is to figure out if the pattern is random.
Let’s make this more concrete. Here are two patterns, from Steven Pinker’s book, The Better Angels of our Nature. One of the patterns is randomly generated. The other imitates a pattern from nature. Can you tell which is which?
pinker-glow-worms-and-stars-plot
Thought about it?
Here is Pinker’s explanation.
The one on the left, with the clumps, strands, voids, and filaments (and perhaps, depending on your obsessions, animals, nudes, or Virgin Marys) is the array that was plotted at random, like stars. The one on the right, which seems to be haphazard, is the array whose positions were nudged apart, like glowworms
That’s right, glowworms. The points on the right records the positions of glowworms on the ceiling of the Waitomo cave in New Zealand. These glowworms aren’t sitting around at random, they’re competing for food, and nudging themselves away from each other. They have a vested interest against clumping together.
Update: Try this out for yourself. After reading this article, praptak and roryokane over at hacker news wrote a script that will generate random and uniform distributions in your browser, nicely illustrating the point.
Try to uniformly sprinkle sand on a surface, and it might look like the pattern on the right. You’re instinctively avoiding places where you’ve already dropped sand. Random processes have no such prejudices, the grains of sand simply fall where they may, clumps and all. It’s more like sprinkling sand with your eyes closed. They key difference is that randomness is not the same thing as uniformity. True randomness can have clusters, like the constellations that we draw into the night sky.
Here’s another example. Imagine a professor asks her students to flip a coin 100 times. One student diligently did the work, and wrote down their results. The other student is a bit of a slacker, and decided to make up fake coin tosses instead of doing the experiment. Can you identify which student is the slacker?
Student 1:
THHHTHTTTTHTTHTTTHHTHTTHT
HHHTHTHHTHTTHHTTTTHTTTHTH
TTHHTTTTTTTTHTHHHHHTHTHTH
THTHTHHHHHTHHTTTTTHTTHHTH
Student 2:
HTTHTTHTHHTTHTHTHTTHHTHTT
HTTHHHTTHTTHTHTHTHHTTHTTH
THTHTHTHHHTTHTHTHTHHTHTTT
HTHHTHTHTHTHHTTHTHTHTTHHT
Take a moment to reason this through.
The first student’s data has clusters – long runs of up to eight tails in a row. This might look surprising, but it’s actually what you’d expect from random coin tosses (I should know – I did a hundred coin tosses to get that data!) The second student’s data in suspiciously lacking in clusters. In fact, in a hundred coin tosses, they didn’t get a single run of four or more heads or tails in a row. This has about a 0.1% chance of ever happening, suggesting that the student fudged the data (and indeed I did).
Image by Scott Adams.
Trying to work out whether a pattern of numbers is random may seem like an arcane mathematical game, but this couldn’t be further from the truth. The study of random fluctuations has its roots in nineteenth century French criminal statistics. As France was rapidly urbanizing, population densities in cities began to shoot up, and crime and poverty became pressing social problems.
371px-Adolphe_Quételet_by_Joseph-Arnold_Demannez
In 1825, France began to collect statistics on criminal trials. What followed was perhaps the first instance of statistical analysis used to study a social problem. Adolphe Quetelet was a Belgian mathematician, and one of the early pioneers of the social sciences. His controversial goal was to apply probability ideas used in astronomy to understand the laws that govern human beings.
In the words of Michael Maltz,
In finding the same regularity in crime statistics that was found in astronomical observations, he argued that, just as there was a true location of a star (despite the variance in the location measurements), there was a true level of criminality: he posited the construct of l’homme moyen (the “average man”) and, moreover, l’homme moyen moral. Quetelet asserted that the average man had a statistically constant “penchant for crime,” one that would permit the “social physicist” to calculate a trajectory over time that “would reveal simple laws of motion and permit prediction of the future” (Gigerenzer et al, 1989).
Quetelet noticed that the conviction rate of criminals was slowly falling over time, and deduced that there must be a downward trend in the “penchant for crime” in French citizens. There were some problems with the data he used, but the essential flaw in his method was uncovered by the brilliant French polymath and scientist Siméon-Denis Poisson.
Simeon_Poisson
Poisson’s idea was both ingenious and remarkably modern. In today’s language, he argued that Quetelet was missing a model of his data. He didn’t account for how jurors actually came to their decisions. According to Poisson, jurors were fallible. The data that we observe is the rate of convictions, but what we want to know is the probability that a defendant is guilty. These two quantities aren’t the same, but they can be related. The upshot is that when you take this process into account, there is a certain amount of variation inherent in conviction rates, and this is what one sees in the French crime data.
In 1837, Poisson published this result in “Research on the Probability of Judgments in Criminal and Civil Matters“. In that work, he introduced a formula that we now call the Poisson distribution. It tells you the odds that a large number of infrequent events result in a specific outcome (such as the majority of French jurors coming to the wrong decision). For example, let’s say that on average, 45 people are struck by lightning in a year. Feed this in to Poisson’s formula, along with the population size, and it will spit out the odds that, say, 10 people will be struck by lightning in a year, or 50, or a 100. The assumption is that lightning strikes are independent, rare events that are just as likely to occur at any time. In other words, Poisson’s formula can tell you the odds of seeing unusual events, simply due to chance.
By Randall Munroe
One of the first applications of Mr. Poisson’s formula came from an unlikely place. Leap sixty years ahead, over the Franco-Prussian war, and land in 1898 Prussia. Ladislaus Bortkiewicz, a Russian statistician of Polish descent, was trying to understand why, in some years, an unusually large number of soldiers in the Prussian army were dying due to horse-kicks. In a single army corp, there were sometimes 4 such deaths in a single year. Was this just coincidence?
A single incidence of death by horse kick is rare (and assumedly independent, unless the horses have a hidden agenda). Bortkiewicz realized that he could use Poisson’s formula to work out how many deaths you expect to see. Here is the prediction, next to the real data.
Number of Deaths by Horse Kick in a year Predicted Instances (Poisson) Observed Instances
0 108.67 109
1 66.29 65
2 20.22 22
3 4.11 3
4 0.63 1
5 0.08 0
6 0.01 0
See how well they line up? The sporadic clusters of horse-related deaths are just what you would expect if horse-kicking was a purely random process. Randomness comes with clusters.
By Ryan North
I decided to try this out for myself. I looked for publicly available datasets for deaths due to rare events, and came across the International Shark Attack File, that tabulates worldwide incidents of sharks attacking people. Here’s the data of shark attacks in South Africa.
Year Number of Shark Attacks in South Africa
2000 4
2001 3
2002 3
2003 2
2004 5
2005 4
2006 4
2007 2
2008 0
2009 6
2010 7
2011 5
The numbers are fairly low, with an average of 3.75. But compare 2008 and 2009. One year has zero shark attacks, and the next has 6. And then in 2010, there are 7. You can already imagine the headlines crying out, “Attack of the sharks!“. But is there really a shark rebellion, or would you expect to see these clusters of shark attacks due to chance? To find out, I compared the data to Mr. Poisson’s prediction.
shark_attacks_south_africa
“Anyone else see the shark fin?” Nice catch, by @Gareth_Elms
In blue are the observed counts of years with a 0,1,2,3.. shark attacks. For example, the long blue bar represents the 3 years in which there were 4 shark attacks (2000, 2005 and 2006). The red dotted line is the Poisson distribution, and it represents the outcomes that you would expect if the shark attacks were a purely random process. It fits the data well – I found no evidence of clustering beyond what is expected by a Poisson process (p=0.87). I’m afraid this rules out the great South African shark uprising of 2010. The lesson, again, is that randomness isn’t uniform.
Which brings us back to the buzzbombs. Here’s a visualization of the number of bombs dropped over different parts, reconstructed by Charles Franklin using the original maps in the British Archives in Kew.
london buzzbomb distribution
Note: A clarification. The plot above shows the distribution of bombs that were dropped over London. The question I’m asking is, if you zoom in to the part of the city most heavily under attack (essentially the mountain that you see in the figure above), are the bombs being guided more precisely, to hit specific targets?
It’s far from a uniform distribution, but does it show evidence of precise targeting? At this point, you can probably guess how to answer this question. In a report titled An Application of the Poisson Distribution, a British statistician named R. D. Clarke wrote,
During the flying-bomb attack on London, frequent assertions were made that the points of impact of the bombs tended to be grouped in clusters. It was accordingly decided to apply a statistical test to discover whether any support could be found for this allegation.
Clarke took a 12 km x 12 km heavily bombed region of South London, and sliced it up in to a grid. In all, he divided it into 576 squares, each about the size of 25 city blocks. Next, he counted the number of squares with 0 bombs dropped, 1 bomb dropped, 2 bombs dropped, and so on.
In all, 537 bombs fell over these 576 squares. That’s a little under one bomb falling per square, on average. He plugged this number into Poisson’s formula, to work out how much clustering you would expect to see by chance. Here’s the relevant table from his paper:
poisson table buzzbombs
Compare the two columns, and you can see how incredibly close the prediction comes to reality. There are 7 squares that were hit by 4 bombs each – but this is what you would expect by chance. Within a large area of London, the bombs weren’t being targeted. They rained down at random in a devastating, city-wide game of Russian roulette.
The Poisson distribution has a habit of creeping up in all sorts of places, some inconsequential, and others life-altering. The number of mutations in your DNA as your cells age. The number of cars ahead of you at a traffic light, or patients in line before you at the emergency room. The number of typos in each of my blog posts. The number of patients with leukemia in a given town. The numbers of births and deaths, marriages and divorces, or suicides and homicides in a given year. The number of fleas on your dog.
From mundane moments to matters of life and death, these Victorian scientists have taught us that randomness plays a larger role in our lives than we care to admit. Sadly, this fact offers little consolation when the cards in life fall against your favor.
“So much of life, it seems to me, is determined by pure randomness.” – Sidney Poitier
References
Shark attacks and the Poisson approximation. A nice introduction to using Poisson’s formula, with applications including the birthday paradox, one of my favorite examples of how randomness is counter-intuitive.
From Poisson to the Present: Applying Operations Research to Problems of Crime and Justice. A good read about the birth of operations research as applied to crime.
Applications of the Poisson probability distribution. Includes a list of many applications of the Poisson distribution.
Steven Pinker’s book The Better Angels of our Nature has many great examples of how our intuition about randomness is generally wrong.
Want to know more about the accuracy of the flying bombs? The story is surprisingly rich, involving counterintelligence and espionage. Here’s a teaser.
____________________________________
Flight over Wall St
19 April 1997
By John Casti
Santa Fe, New Mexico
ON 29 March 1900, in a dusty seminar room at the Sorbonne in Paris, the renowned mathematician Jules Henri Poincaré was presiding as a student defended a slightly unusual doctoral dissertation. In Poincaré`s words, the topic was “somewhat remote from those our candidates are in the habit of treating”. This topic was a mathematical treatment of how prices for French government bonds and their options fluctuated on the Paris Bourse. The author of this dissertation, Louis Bachelier, received the insultingly low grade of mention honorable rather than the more usual mention très honorable, the level needed to be taken seriously as a candidate for an academic post in France. Perhaps the dissertation’s title, The Theory of Speculation, as well as its frankly commercial character had something to do with the disdain with which the examiners viewed Bachelier’s efforts. Who can say? What we do know is that this was pioneering work, the academic community’s initial salvo in the battle to unlock the secrets of speculative markets. And Bachelier’s ideas were truly visionary, for a century later his conclusions remain at the heart of the dominant paradigm as to how prices fluctuate in such markets.
[changes in prices]
Bachelier focused his mathematical artillery on changes in prices, rather than on the prices themselves. Prices sometimes go up, of course, and sometimes they go down. But by studying the ups and downs with care, Bachelier discovered that he could say more than that.
[The ups and downs]
First, he noticed that price changes over one interval have nothing to do with changes over another. If a stock price goes …
Read more: https://www.newscientist.com/article/mg15420784-700-flight-over-wall-st/
source:
https://www.newscientist.com/article/mg15420784-700-flight-over-wall-st/
https://www.newscientist.com/article/mg15420784.700-flight-over-wall-st.html/
____________________________________
Mario Livio, Brilliant blunders, 2013 [ ]
p.210
Penzias and Wilson were working at the Bell Telephone Laboratories in New Jersey with an antenna built for communication satellites. To their annoyance, they were picking up some sort of pervasive background radio noise: microwave radiation that appeared to be the same from all directions. After failing to explain away this disturbing “hiss” as an instrumental artifact, Penzias and Wilson finally announced the detection of an intergalatic temperature excess of about 3 Kelvin (3 degrees above absolute zero).
p.210
Lacking the necessary background, Penzias and Wilson did not realize initially what they had found. Robert Dicke of Princeton University, however, recognized the signal immediately. Dicke was in the process of building a radiometer to search for the relic radiation from the big bang, previously predicted by Alpher, Hermann, and Gamow.
p.210
Consequently, his correct interpretation of the results of Penzias and Wilson literally transformed the big bang theory from hypothesis into experimentally tested physics.
p.210
As the universe expanded, the incredibly hot, dense, and opaque fireball coolded down continuously, eventually reaching its present temperature of about 2.7 Kelvin.
p.210
Since then, observations of the cosmic microwave background have produced some of the most precise measurement in cosmology.
p.210
its intensity changes with wavelength precisely as expected from a thermal source
(Brilliant blunders: from Darwin to Einstein ─ colossal mistakes by great scientists that changed our understanding of life and the universe / Mario Livio., 1. errors, scientific., Q172.5.E77L58 2013, 500─dc23, first Simon & Schuster hardcover edition May 2013, 2013, )
____________________________________
Kevin Kelly, out of control, 1994
p.315
“Natural selection is the editor, not the author,” says Lynn Margulis.
p.316
the Krebs cycle is the basic fuel plant in every cell in your body. It has worked fine for hundreds of millions of years. There is simply too little to gain, and far too much to lose, in fiddling with it now. When a variation is detected in the code for the Krebs cycle, it is quickly extinguished. On the other hand, body size and body proportions might be worth tweaking; let's leave that area open to variation.
p.317
“Oh, randomness is just an excuse for ignorance”, quips Lynn Margulis.
(Kevin Kelly, out of control, 1994, filename: ooc-mf.pdf )
____________________________________
Beer, 1959 (Stafford Beer)
*** [ Stafford Beer's classification of systems, p.171, George P. Richardson, 1991, Feedback thought in social science and systems theory ]
Stafford Beer's classification of systems
[p.171]
-----------------------------------------------------------------------
SYSTEMS Simple Complex Exceedingly
complex
-----------------------------------------------------------------------
Deterministic Window catch Electronic digital EMPTY
computer
--------------------------------------
Billiards Planetary system
--------------------------------------
Machine-shop Automation
lay-out
-----------------------------------------------------------------------
Probabilistic Penny tossing Stockholding The economy
---------------------------------------------------------
Jellyfish Conditioned The brain
movement reflexed
---------------------------------------------------------
Statistical Industrial THE COMPANY
quality control profitability
FIGURE 4.1: Stafford Beer's classification of systems based on degrees of complexity and uncertainty. Source: Beer (1959, p. 18).
Beer, Stafford (1959/1967). Cybernetics and Management (London: English Universities Press).
(Richardson, George P., Feedback thought in social science and systems theory, copyright © 1991 by the University of Pennsylvania Press)
(Feedback thought in social science and systems theory / George P. Richardson (1991), 1. social science--methodology., 2. feedback control systems., p.171 )
____________________________________
BANK OF ENGLAND
Speech
Tails of the unexpected
Paper by Andrew G Haldane (and) Benjamin Nelson
Given at “The Credit Crisis Five Years On: Unpacking the Crisis”, conference held at the University of Edinburg Business School, 8-9 June
8 June 2012
•••• ••• ••••
But as Nassim Taleb reminded us, it is possible to be Fooled by Randomness (Taleb, 2001). For Taleb, the origin of this mistake was the ubiquity in economics and finance of a particular way of describing the distribution of possible real world outcomes. For non-nerds, this distribution is often called the bell-curve. For nerds, it is the normal distribution. For nerds who like to show-off, the distribution is Gaussian.
[normal distribution, bell-curve, Gaussian distribution]
The normal distribution provides a beguilingly simple description of the world. Outcomes lie symmetrically around the mean, with a probability that steadily decays. It is well-known that repeated games of chance deliver random outcomes in line with this disbribution: tosses of a fair coin, sampling of coloured balls from a jam-jar, bets on a lottery number, games of paper/scissors/stone. Or have you been fooled by randomness?
•••• ••• ••••
All speeches are available online at
http://www.bankofengland.co.uk/publications/Pages/speeches/default.aspx
<put additional URL here>
https://drive.google.com/file/d/1A18Z9AkjaUqQFs4HX_j32NqhU7IlrKPU/view?usp=sharing
https://drive.google.com/file/d/1A18Z9AkjaUqQFs4HX_j32NqhU7IlrKPU/
____________________________________
- probabilities, statistics, distribution curve
- probabilities table, statistics table, look-up table
- distribution table, frequency table, data table
- [normal distribution, bell-curve, Gaussian distribution]
- In statistics, curves like that are called “long-tailed distribution”, because the tail of the curve is very long relative to the head., (p.10, Chris Anderson, The long tail, 2006)
____________________________________
Technology
Computer Sciences
October 16, 2015
System that replaces human intuition with algorithms outperforms human teams
by Larry Hardesty, Massachusetts Institute of Technology
http://phys.org/news/2015-10-human-intuition-algorithms-outperforms-teams.html
http://groups.csail.mit.edu/EVO-DesignOpt/groupWebSite/uploads/Site/DSAA_DSM_2015.pdf
Big-data analysis consists of searching for buried patterns that have some kind of predictive power. But choosing which "features" of the data to analyze usually requires some human intuition.
"What we observed from our experience solving a number of data science problems for industry is that one of the very critical steps is called feature engineering," Veeramachaneni says. "The first thing you have to do is identify what variables to extract from the database or compose, and for that, you have to come up with a lot of ideas."
____________________________________
____________________________________
No comments:
Post a Comment