Product Defects

NOTE: This was completed as part of the Data Science Foundations course on CodeAcademy. The template was provided by CodeAcademy.

You are in charge of monitoring the number of products from a specific factory. You’ve been told that the number of defects on a given day follows the Poisson distribution with the rate parameter (lambda) equal to 7. You’re new here, so you want to get a feel for what it means to follow the Poisson(7) distribution. You remember that the Poisson distribution is special because the rate parameter represents the expected value of the distribution, so in this case, the expected value of the Poisson(7) distribution is 7 defects per day.

You will investigate certain attributes of the Poisson(7) distribution to get an intuition for how many defective objects you should expect to see in a given amount of time. You will also practice and apply what you know about the Poisson distribution on a practice data set that you will simulate yourself.

Distribution in Theory

1. Create a variable called lam that represents the rate parameter of our distribution.

from scipy.stats import poisson
import matplotlib.pyplot as plt
import numpy as np
## Histogram function
def histogram_function(rand_vars):
    plt.hist(rand_vars, bins = np.arange(len(set(rand_vars)))-0.5, edgecolor = "black")
    plt.xticks(list(range(rand_vars.max())))
    plt.show()

## Task 1:
lam = 7

1. You know that the rate parameter of a Poisson distribution is equal to the expected value. So in our factory, the rate parameter would equal the expected number of defects on a given day. You are curious about how often we might observe the exact expected number of defects.

   Calculate and print the probability of observing exactly lam defects on a given day.

## Task 2 - Observing lam defects in a given day
print(poisson.pmf(lam,lam))

0.14900277967433773

1. Our boss said that having 4 or fewer defects on a given day is an exceptionally good day. You are curious about how often that might happen.

   Calculate and print the probability of having one of those days.

## Task 3 - Observing 4 or fewer defects in a given day
print(poisson.cdf(4, lam))

0.17299160788207146

1. On the other hand, our boss said that having more than 9 defects on a given day is considered a bad day.

   Calculate and print the probability of having one of these bad days.

## Task 4 - more than 9 defects on a given day
print(poisson.sf(9, lam))

0.16950406276132673

Distribution in Practice

1. You’ve familiarized yourself a little bit about how the Poisson distribution works in theory by calculating different probabilities. But let’s look at what this might look like in practice.

   Create a variable called year_defects that has 365 random values from the Poisson distribution.

## Task 5 - random values in the Poisson Distribution
year_defects = np.array(poisson.rvs(lam, size=365))
histogram_function(year_defects)

1. Let’s take a look at our new dataset. Print the first 20 values in this dataset.

## Task 6: - First 20 values in dataset
year_defects[0: 20]

array([ 5,  5,  8,  4,  6, 11,  7,  8,  7, 11,  3,  7,  6, 10,  5,  5,  5,
       13,  7,  4])

1. If we expect 7 defects on a given day, what is the total number of defects we would expect over 365 days?

   Calculate and print this value to the output terminal.

## Task 7 - total number of defects in the year
print(lam *365)

2555

1. Calculate and print the total sum of the dataset year_defects. How does this compare to the total number of defects we expected over 365 days?

## Task 8 - total sum of dataset
sum = np.sum(year_defects)
print(sum)

2568

1. Calculate and print the average number of defects per day from our simulated dataset.

   How does this compare to the expected average number of defects each day that we know from the given rate parameter of the Poisson distribution?

## Task 9 - average of simulated dataset
average = np.mean(year_defects)
print(average)

7.035616438356165

1. You’re worried about what the highest amount of defects in a single day might be because that would be a hectic day.

   Print the maximum value of year_defects.

## Task 10 - highest number of days in a given day
maximum = year_defects.max()
print(maximum)

18

1. Wow, it would probably be super busy if there were that many defects on a single day. Hopefully, it is a rare event!

   Calculate and print the probability of observing that maximum value or more from the Poisson(7) distribution.

## Task 11 - probability of observing maximum value or more
print(poisson.sf(maximum-1, lam))

0.000361784316602241

Extra

1. Congratulations! At this point, you have now explored the Poisson distribution and even worked with some simulated data. We have a couple of extra tasks if you would like an extra challenge. Feel free to try them out or move onto the next topic!

   Let’s say we want to know how many defects in a given day would put us in the 90th percentile of the Poisson(7) distribution. One way we could calculate this is by using the following method:

   stats.poisson.ppf(percentile, lambda)
   

   percentile is equal to the desired percentile (a decimal between 0 and 1), and lambda is the lambda parameter of the Poisson distribution. This function is essentially the inverse of the CDF.

   Use this method to calculate and print the number of defects that would put us in the 90th percentile for a given day. In other words, on 90% of days, we will observe fewer defects than this number.

## Task 12:
ninteythPercentile = poisson.ppf(0.9, lam)
print(ninteythPercentile)

10.0

1. Now let’s see what proportion of our simulated dataset year_defects is greater than or equal to the number we calculated in the previous step.

   By definition of a percentile, we would expect 1 - .90, or about 10% of days to be in this range.

   To calculate this:

   1. Count the number of values in the dataset that are greater than or equal to the 90th percentile value.

   2. Divide this number by the length of the dataset.

## Task 13:

print(np.sum(year_defects > poisson.ppf(0.9,lam))/len(year_defects))

0.11232876712328767