pfoeの実装#226
Open
sabakan495 wants to merge 51 commits into
Open
Conversation
Kasaiatsuki
reviewed
Jun 7, 2026
|
|
||
| self.zed_camera.retrieve_image(self.zed_image, sl.VIEW.LEFT) | ||
| image = self.zed_image.get_data() | ||
| resized_image = cv2.resize(image, (224, 224)) |
There was a problem hiding this comment.
224x224で画像を使うなら800x600を圧縮するのではなく真ん中でクロップした方が画像が横に潰れなくて良いのではないでしょうか
Kasaiatsuki
reviewed
Jun 7, 2026
Comment on lines
+112
to
+113
| left_stick = msg.axes[0] | ||
| right_stick = msg.axes[4] |
There was a problem hiding this comment.
左スティックが前後の値、右スティックが左右の値をとることを想定しているなら、
left_stick = msg.axes[1]
right_stick = msg.axes[2]
が正しいと思います。
Kasaiatsuki
reviewed
Jun 7, 2026
| left_stick = msg.axes[0] | ||
| right_stick = msg.axes[4] | ||
|
|
||
| self.joy_value = np.arctan2(right_stick, left_stick) |
There was a problem hiding this comment.
left_stick(並進速度)は記録し始めてからは基本倒しっぱなしで情報があまりないと思うので
self.joy_value = right_stick
のようにして１つにするのはどうでしょうか
left_stickが使わなそうでしたら上のとこも消してください。
Kasaiatsuki
reviewed
Jun 7, 2026
Comment on lines
+117
to
+152
| def timer_callback(self)->None: | ||
| if self.is_paused: | ||
| return | ||
|
|
||
| current_time = time.time() | ||
|
|
||
| if self.sdk_flag_: | ||
| image = self._capture_data_from_zed() | ||
| if image is None or self.joy_value is None: | ||
| return | ||
|
|
||
| if self.last_sample_time is None or current_time - self.last_sample_time >= SAMPLE_INTERVAL: | ||
| get_image_tensor = self.feature_extractor.extract(image).astype(np.float32) | ||
| row = np.concatenate([ | ||
| get_image_tensor, | ||
| np.array([self.joy_value], dtype=np.float32) | ||
| ]) | ||
| self.bin_file.write(row.tobytes()) | ||
| self.log_writer.writerow([self.row_count, self.joy_value]) | ||
| self.row_count += 1 | ||
| self.last_sample_time = current_time | ||
|
|
||
| else: | ||
| if self.latest_image is None or self.joy_value is None: | ||
| return | ||
| if self.last_sample_time is None or current_time - self.last_sample_time >= SAMPLE_INTERVAL: | ||
| cv_image = self.bridge.imgmsg_to_cv2(self.latest_image, desired_encoding='rgb8') | ||
| get_image_tensor = self.feature_extractor.extract(cv_image).astype(np.float32) | ||
| row = np.concatenate([ | ||
| get_image_tensor, | ||
| np.array([self.joy_value], dtype=np.float32) | ||
| ]) | ||
| self.bin_file.write(row.tobytes()) | ||
| self.log_writer.writerow([self.row_count, self.joy_value]) | ||
| self.row_count += 1 | ||
| self.last_sample_time = current_time |
There was a problem hiding this comment.
timer_callbackとサンプルタイムが同じだと画像を10hzで撮ったあとの特徴量の抽出までが間に合わないと思います。
特徴量の抽出がどれくらいかかるのか分かりませんがサンプルタイムを大きくするか特徴量の抽出はデータを取った後にするのがいいと思います。
Kasaiatsuki
reviewed
Jun 7, 2026
| param.requires_grad = False | ||
|
|
||
| self.transform = transforms.Compose([ | ||
| transforms.Resize((224, 224)), |
Kasaiatsuki
reviewed
Jun 7, 2026
Comment on lines
+13
to
+14
| weights = models.EfficientNet_B0_Weights.IMAGENET1K_V1 | ||
| backbone = models.efficientnet_b0(weights=weights) |
There was a problem hiding this comment.
ネットで一度インストールするよりかは事前に重みファイルをディレクトリの中に置いておいてそれを使った方が安心安全だと思います。
Kasaiatsuki
reviewed
Jun 7, 2026
Comment on lines
+8
to
+22
| int ParticleFilter::init(const std::string& data_dir){ | ||
| episodes_.clear(); | ||
|
|
||
| std::vector<std::string> paths; | ||
| for (const auto& entry : std::filesystem::recursive_directory_iterator(data_dir)) { | ||
| if (entry.is_regular_file() && | ||
| entry.path().extension() == ".bin") { // data.bin が引っかかる | ||
| paths.push_back(entry.path().string()); | ||
| } | ||
| } | ||
| if (paths.empty()) { | ||
| std::cerr << "no .bin under: " << data_dir << std::endl; | ||
| return -1; // 1個も無ければ失敗 | ||
| } | ||
| } |
Kasaiatsuki
reviewed
Jun 7, 2026
| right_stick = msg.axes[4] | ||
|
|
||
| self.joy_value = np.arctan2(right_stick, left_stick) | ||
|
|
There was a problem hiding this comment.
今後pfoeで自己位置のエピソードを推定して走行経路の条件をe2eに渡すのなら、分岐路に入るときに左折(2),右折(3)(道なりは１)をジョイコンのボタンで記録できるようにした方がいいと思います。
これは既にfeat/yolop_shortcutやfeat/shannon_surpriseで実装しているので参考にしてください。
|
まだ作成途中だとは思いますがコメント入れました。 |
Author
|
ParticleFilter.cpp，ParticleFilter.hppの一連の流れを実装した |
Author
|
指摘点を修正しました |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
教師走行で記録した「映像＋操作の時系列（エピソード）」を覚えておき、本番走行中に「今カメラに映ってる景色は、記録のどのコマに一番似てるか」をパーティクルフィルタで推定して、そのコマの操作（joy値）を出力する。
学習（ニューラルネットの重み更新）はしない。記録との照合だけで動く方式。