300. Longest Increasing Subsequence
Problem Description
The problem is about finding the length of the longest strictly increasing subsequence within an array of integers named nums
. A subsequence is defined as a sequence that can be derived from another sequence by deleting some or no elements without changing the order of the remaining elements. A strictly increasing subsequence means each element must be larger than the previous one in the subsequence. For example, in [10, 9, 2, 5, 3, 7, 101, 18]
, the longest increasing subsequence is [2, 3, 7, 101]
, and its length is 4.
Intuition
To solve this problem effectively, we can utilize the concept of Dynamic Programming (DP) or Binary Indexed Trees (Fenwick Tree). The intuition behind using these approaches is efficiency in both computation and memory usage.
Using Dynamic Programming to solve this problem would involve creating an array where each element dp[i]
represents the length of the longest increasing subsequence that ends with the i
-th element of the input array. To find the value of dp[i]
, we look at all previous elements j
, where j < i
, and if nums[j] < nums[i]
, that means nums[i]
can extend the subsequence ending with nums[j]
, so we pick the maximum length out of all possible j
.
However, the given solution uses a Binary Indexed Tree (BIT), which is a data structure that helps with efficiently updating elements and querying prefixes in a log-scaled time. The BIT is an advanced optimization usually used to improve the running time complexity from O(N^2) to O(N log N), where N is the number of elements in the input array.
The solution employs coordinate compression technique by sorting the unique elements to reduce the range of numbers, maintaining the original order of elements. This helps us to use the BIT effectively. For every element x
in the compressed coords, we use 'query' to find the longest increasing subsequence that ends before x
and 'update' to extend the subsequence ending at x
if it's longer than the previously recorded one. Thus, we maintain the length of the LIS up to each point in the array leveraging the BIT's efficient update and query capabilities.
Learn more about Binary Search and Dynamic Programming patterns.
Solution Approach
The solution provided uses a Binary Indexed Tree (BIT), also known as a Fenwick Tree, to keep track of the maximum subsequence's length during the iteration of the numbers in nums
. Here's a walkthrough of the implementation:
-
Coordinate Compression: We start by compressing the coordinates of
nums
. This is done by creating a sorted lists
of the unique elements innums
. This step is key because it maps the large or non-consecutive input space into a smaller and consecutive range, which is essential to correctly harness the power of BITs. -
Initialization: We initialize the BIT with the size equal to the number of unique elements in
nums
. The BIT is represented by a listself.c
and supports theupdate
andquery
operations.-
Update: The
update
method updates the BIT with a new element's length of the longest increasing subsequence. It takes an inputx
representing the index in the BIT andv
as the value to update (length of LIS). The method updates the elements in the BIT by comparing and setting the current indexed value with the maximum between the existing value andv
.self.c[x] = max(self.c[x], v)
-
Query: The
query
method retrieves the largest value of the subsequence's length from the BIT up to the indexx
. The result is the length of the longest increasing subsequence considering elements up to the indexx
.mx = max(mx, self.c[x])
-
-
Iteration: For each number
x
in the input listnums
, we do the following:-
Find the index of
x
in the compressed coordinates (the sorted lists
). This index is used to look up and update the BIT. -
Query the BIT for the index just below the current element (i.e.,
x - 1
). The BIT returns the length of the longest subsequence beforex
. -
Increment the length by 1, which represents that
x
can extend the subsequence. -
Update the answer holding the maximum length observed so far, and also update the BIT at the position
x
with the new length of the subsequence.
-
-
Result: Once the iteration is completed, the variable
ans
holds the length of the longest increasing subsequence.
The beauty of this approach lies in its efficiency. While a straightforward Dynamic Programming approach would have a time complexity of O(N^2)
, using a BIT brings it down to O(N log N)
because both update
and query
operations on a BIT are performed in O(log N)
time.
In conclusion, the reference approach mentioned that one can use either Dynamic Programming or a Binary Indexed Tree. The solution provided implements the latter for an efficient resolution of the Longest Increasing Subsequence problem.
Ready to land your dream job?
Unlock your dream job with a 2-minute evaluator for a personalized learning plan!
Start EvaluatorExample Walkthrough
Let's consider an input array nums
with the following integers: [3, 1, 5, 2, 6]
.
-
Coordinate Compression:
- We begin by sorting the unique elements of
nums
:[1, 2, 3, 5, 6]
. - Next, we map them to their indices in this sorted array:
1 -> 0
,2 -> 1
,3 -> 2
,5 -> 3
,6 -> 4
.
- We begin by sorting the unique elements of
-
Initialization:
- We then initialize a Binary Indexed Tree (BIT) to keep track of the maximum length of the increasing subsequence. Since we have 5 unique numbers, the BIT size will be 5.
-
Iteration:
- For each number in
nums
:- We find the index in the sorted array:
3 -> 2
,1 -> 0
,5 -> 3
,2 -> 1
,6 -> 4
. - We query the BIT for the index just below the current element's index to find the longest subsequence before it and then add 1 for the current element. We also update the BIT at that index.
- Iterative Steps:
- For
3
: Query BIT for index1
, get0
. Update BIT at index2
with0+1 = 1
. - For
1
: Query BIT for index-1
, get0
. Update BIT at index0
with0+1 = 1
. - For
5
: Query BIT for index2
, get1
. Update BIT at index3
with1+1 = 2
. - For
2
: Query BIT for index0
, get1
. Update BIT at index1
with1+1 = 2
. - For
6
: Query BIT for index3
, get2
. Update BIT at index4
with2+1 = 3
.
- For
- We find the index in the sorted array:
- For each number in
-
Result:
- The last update indicates that the longest increasing subsequence has a length of
3
, which corresponds to either[1, 2, 6]
or[1, 5, 6]
. Hence, the result for this example is3
.
- The last update indicates that the longest increasing subsequence has a length of
By using coordinate compression, we efficiently map our problem into a smaller space where the BIT can operate. With each iteration updating and querying the BIT, we maintain optimal running time. This approach significantly reduces the computational complexity from O(N^2)
in the native dynamic programming approach to O(N log N)
.
Solution Implementation
1from bisect import bisect_left
2
3class BinaryIndexedTree:
4 def __init__(self, size: int):
5 # Initialize the size of the binary indexed tree and the storage array
6 self.size = size
7 self.tree = [0] * (size + 1)
8
9 def update(self, index: int, value: int):
10 # Update the tree with the given value at the given index
11 while index <= self.size:
12 self.tree[index] = max(self.tree[index], value)
13 index += index & -index
14
15 def query(self, index: int) -> int:
16 # Query the max value up to the given index
17 max_value = 0
18 while index:
19 max_value = max(max_value, self.tree[index])
20 index -= index & -index
21 return max_value
22
23
24class Solution:
25 def length_of_lis(self, nums: List[int]) -> int:
26 # Sort and deduplicate the input numbers to map the values to tree indices
27 sorted_unique_nums = sorted(set(nums))
28 tree = BinaryIndexedTree(len(sorted_unique_nums))
29
30 # Initialize the answer to the length of the LIS found so far
31 longest_increasing_subseq_length = 1
32
33 # Process each number in the input list
34 for num in nums:
35 # Get the index of the number in the sorted unique array, plus one
36 tree_index = bisect_left(sorted_unique_nums, num) + 1
37
38 # Query the tree for the current length of LIS ending with a number less than the current one
39 length_up_to_num = tree.query(tree_index - 1) + 1
40
41 # Update the answer with the maximal length found
42 longest_increasing_subseq_length = max(longest_increasing_subseq_length, length_up_to_num)
43
44 # Update the tree for this index with the current length of LIS
45 tree.update(tree_index, length_up_to_num)
46
47 # Return the length of the longest increasing subsequence
48 return longest_increasing_subseq_length
49
1class Solution {
2 public int lengthOfLIS(int[] nums) {
3 // Clone nums array and sort it to remove duplicates and to have a sorted reference array
4 int[] sortedArray = nums.clone();
5 Arrays.sort(sortedArray);
6
7 // Remove duplicates in the sorted array to construct a unique set of numbers
8 int uniqueNumsCount = 0;
9 int length = sortedArray.length;
10 for (int i = 0; i < length; ++i) {
11 if (i == 0 || sortedArray[i] != sortedArray[i - 1]) {
12 sortedArray[uniqueNumsCount++] = sortedArray[i];
13 }
14 }
15
16 // Initialize a Binary Indexed Tree, or Fenwick Tree
17 BinaryIndexedTree tree = new BinaryIndexedTree(uniqueNumsCount);
18
19 // Initialize the answer with 1 considering a single element will have LIS as 1
20 int maxLISLength = 1;
21
22 // Iterate over the original array and update Fenwick Tree for each number
23 for (int num : nums) {
24 int index = binarySearch(sortedArray, num, uniqueNumsCount);
25 int currentLISLength = tree.query(index - 1) + 1;
26 maxLISLength = Math.max(maxLISLength, currentLISLength);
27 tree.update(index, currentLISLength);
28 }
29 return maxLISLength;
30 }
31
32 private int binarySearch(int[] nums, int target, int rightBound) {
33 // Perform binary search for the position of target in nums within bounds [0, rightBound]
34 int left = 0;
35 while (left < rightBound) {
36 int mid = (left + rightBound) >> 1;
37 if (nums[mid] >= target) {
38 rightBound = mid;
39 } else {
40 left = mid + 1;
41 }
42 }
43 // Return the position index of the target incremented by 1 to adjust for BIT index
44 return left + 1;
45 }
46}
47
48class BinaryIndexedTree {
49 private int size;
50 private int[] tree;
51
52 public BinaryIndexedTree(int size) {
53 this.size = size;
54 tree = new int[size + 1];
55 }
56
57 public void update(int index, int value) {
58 // Update the tree with new value at the given index
59 // (BIT index is 1-based, hence increment the input index by 1)
60 while (index <= size) {
61 tree[index] = Math.max(tree[index], value);
62 index += index & -index;
63 }
64 }
65
66 public int query(int index) {
67 // Query the largest value up to the given index
68 int maxValue = 0;
69 while (index > 0) {
70 maxValue = Math.max(maxValue, tree[index]);
71 index -= index & -index;
72 }
73 return maxValue;
74 }
75}
76
1#include <vector>
2#include <algorithm>
3
4using namespace std;
5
6// BinaryIndexedTree (a.k.a. Fenwick Tree) helps to perform efficient
7// range queries and updates on an array of numbers.
8class BinaryIndexedTree {
9public:
10 // Constructor initializes the tree with the given size.
11 explicit BinaryIndexedTree(int size)
12 : size_(size)
13 , tree_(size + 1) {}
14
15 // Update the tree by setting the maximum value at position x to v.
16 void Update(int x, int v) {
17 while (x <= size_) {
18 tree_[x] = max(tree_[x], v); // Ensure we only store the maximum value.
19 x += x & -x; // Move to the next index to update.
20 }
21 }
22
23 // Query the maximum value from 1 to x in the tree.
24 int Query(int x) const {
25 int max_value = 0;
26 while (x > 0) {
27 max_value = max(max_value, tree_[x]); // Update the max_value with the maximum up to index x.
28 x -= x & -x; // Move to the preceding elements' index.
29 }
30 return max_value;
31 }
32
33private:
34 int size_; // The size of the tree.
35 vector<int> tree_; // The tree represented as a vector where the tree's values are stored.
36};
37
38// Solution class contains the method lengthOfLIS which computes
39// the length of the Longest Increasing Subsequence in an array of numbers.
40class Solution {
41public:
42 // Computes the length of the longest increasing subsequence.
43 int lengthOfLIS(vector<int>& nums) {
44 vector<int> sortedNums = nums;
45
46 // Sort the elements and remove duplicates to compress the values.
47 sort(sortedNums.begin(), sortedNums.end());
48 auto lastUnique = unique(sortedNums.begin(), sortedNums.end());
49 sortedNums.erase(lastUnique, sortedNums.end());
50
51 // Initialize the Binary Indexed Tree with the size of the unique sorted list.
52 BinaryIndexedTree tree(sortedNums.size());
53
54 int longest = 1; // Base case: the minimum length of LIS is 1.
55
56 // Iterating through each number in nums.
57 for (int num : nums) {
58 // Map the number to its index in the sorted array (1-indexed).
59 int index = lower_bound(sortedNums.begin(), sortedNums.end(), num) - sortedNums.begin() + 1;
60
61 // Query the tree to find the length of the LIS ending before the current number.
62 int prevLIS = tree.Query(index - 1) + 1; // Adding 1 includes the current number in the sequence.
63
64 // Update the answer with the maximum LIS found so far.
65 longest = max(longest, prevLIS);
66
67 // Update the tree with the LIS for this index.
68 tree.Update(index, prevLIS);
69 }
70
71 // Return the length of the longest increasing subsequence.
72 return longest;
73 }
74};
75
1// Number of elements handled by the tree
2let treeSize: number;
3// BIT data structure array
4let bitTree: number[];
5
6// Initialize the Binary Indexed Tree with a given size
7function initializeBIT(size: number) {
8 treeSize = size;
9 bitTree = new Array(size + 1).fill(0);
10}
11
12// Update the BIT with a value at a specific index
13function updateBIT(index: number, value: number) {
14 // Loop until end of tree array
15 while (index <= treeSize) {
16 // Update maximum value for index
17 bitTree[index] = Math.max(bitTree[index], value);
18 // Move to next index
19 index += index & -index;
20 }
21}
22
23// Query the maximum value up to a specific index
24function queryBIT(index: number): number {
25 let maxVal = 0;
26 // Accumulate maximum values from the tree
27 while (index) {
28 maxVal = Math.max(maxVal, bitTree[index]);
29 // Move to previous index
30 index -= index & -index;
31 }
32 return maxVal;
33}
34
35// Find the length of the Longest Increasing Subsequence (LIS) in an array
36function lengthOfLIS(nums: number[]): number {
37 // Remove duplicates and sort the elements
38 const uniqueSortedNums = [...new Set(nums)].sort((a, b) => a - b);
39 // Initialize the BIT with the size of unique elements
40 initializeBIT(uniqueSortedNums.length);
41 let ans = 1;
42
43 for (let num of nums) {
44 // Map current number to its index in the sorted array
45 let mappedIndex = binarySearch(uniqueSortedNums, num);
46 // Query the maximum value found before the current number's index
47 const maxVal = queryBIT(mappedIndex - 1) + 1;
48 // Update answer with the maximum value found
49 ans = Math.max(ans, maxVal);
50 // Update the BIT with the new value
51 updateBIT(mappedIndex, maxVal);
52 }
53
54 return ans;
55}
56
57// Perform binary search to find the index of a given number 'target' in 'nums'
58function binarySearch(nums: number[], target: number): number {
59 let left = 0;
60 let right = nums.length - 1;
61 while (left < right) {
62 // Go to the middle index
63 const mid = (left + right) >> 1;
64 // Narrow down left or right boundary
65 if (nums[mid] >= target) {
66 right = mid;
67 } else {
68 left = mid + 1;
69 }
70 }
71 // Convert 0-based index to 1-based for BIT operations
72 return left + 1;
73}
74
Time and Space Complexity
Time Complexity
The lengthOfLIS
function has several components to analyze:
-
Sorting the set of unique elements:
O(U * log(U))
, whereU
is the number of unique elements innums
. -
Binary indexed tree operations:
- Each
update
operation isO(log(N))
, whereN
is the length of the unique set after sorting. - Each
query
operation is alsoO(log(N))
.
Since both tree operations occur inside a loop that runs once for each element in
nums
, the complexity for these isO(N * log(N))
, where "N" here refers to the number of elements innums
. - Each
Therefore, combining sorting with the tree operations, the overall time complexity is O(U * log(U) + N * log(N))
. Since the set of unique elements U
cannot be larger than the total number of elements N
, the time complexity simplifies to O(N * log(N))
.
Space Complexity
- The space for the sorted set of unique elements:
O(U)
. - The space for the
BinaryIndexedTree
class, which includes an arrayc
:O(N + 1)
for the+ 1
is appended to handle 1-based indexing.
The overall space complexity thus is the larger of the two, which is O(N)
due to the binary indexed tree array adding an element for 1-based indexing.
Learn more about how to find time and space complexity quickly using problem constraints.
Consider the classic dynamic programming of longest increasing subsequence:
Find the length of the longest subsequence of a given sequence such that all elements of the subsequence are sorted in increasing order.
For example, the length of LIS for [50, 3, 10, 7, 40, 80]
is 4
and LIS is
[3, 7, 40, 80]
.
What is the recurrence relation?
Recommended Readings
Binary Search Speedrun For each of the Speedrun questions you will be given a binary search related problem and a corresponding multiple choice question The multiple choice questions are related to the techniques and template s introduced in the binary search section It's recommended that you have gone through at
What is Dynamic Programming Prerequisite DFS problems dfs_intro Backtracking problems backtracking Memoization problems memoization_intro Pruning problems backtracking_pruning Dynamic programming is an algorithmic optimization technique that breaks down a complicated problem into smaller overlapping sub problems in a recursive manner and uses solutions to the sub problems to construct a solution
LeetCode Patterns Your Personal Dijkstra's Algorithm to Landing Your Dream Job The goal of AlgoMonster is to help you get a job in the shortest amount of time possible in a data driven way We compiled datasets of tech interview problems and broke them down by patterns This way we
Want a Structured Path to Master System Design Too? Don’t Miss This!